Package: osxcollector
OSXCollector ManualOSXCollector is a forensic evidence collection & analysis toolkit for OSX.Forensic CollectionThe collection script runs on a potentially infected machine and outputs a JSON file that describes the target machine. OSXCollector gathers information from plists, SQLite databases and the local file system.Forensic AnalysisArmed with the forensic collection, an analyst can answer the question like:Is this machine infected?How'd that malware get there?How can I prevent and detect further infection?Yelp automates the analysis of most OSXCollector runs converting its output into an easily readable and actionable summary ofjust the suspicious stuff. Check outOSXCollector Output Filters projectto learn how to make the most of the automated OSXCollector output analysis.Performing Collectionosxcollector.pyis a single Python file that runs without any dependencies on a standard OSX machine. This makes it really easy to run collection on any machine - no fussing with brew, pip, config files, or environment variables. Just copy the single file onto the machine and run it:sudo osxcollector.pyis all it takes.$sudoosxcollector.py
Wrote35394lines.
Outputinosxcollect-2014_12_21-08_49_39.tar.gzIf you have just cloned the GitHub repository,osxcollector.pyis insideosxcollector/directory, so you need to run it as:$sudoosxcollector/osxcollector.pyIMPORTANT:please make sure thatpythoncommand on your Mac OS X machine uses the default Python interpreter shipped with the system and is not overridden, e.g. by the Python version installed through brew. OSXCollector relies on a couple of native Python bindings for OS X libraries, which might be not available in other Python versions than the one originally installed on your system.
Alternatively, you can runosxcollector.pyexplicitly specifying the Python version you would like to use:$sudo/usr/bin/python2.7osxcollector/osxcollector.pyThe JSON output of the collector, along with some helpful files like system logs, has been bundled into a .tar.gz for hand-off to an analyst.osxcollector.pyalso has a lot of useful options to change how collection works:-i INCIDENT_PREFIX/--id=INCIDENT_PREFIX:
Sets an identifier which is used as the prefix of the output file. The default value isosxcollect.$sudoosxcollector.py-iIncontinentSealord
Wrote35394lines.
OutputinIncontinentSealord-2014_12_21-08_49_39.tar.gzGet creative with incident names, it makes it easier to laugh through the pain.-p ROOTPATH/--path=ROOTPATH:
Sets the path to the root of the filesystem to run collection on. The default value is/. This is great for running collection on the image of a disk.$sudoosxcollector.py-p'/mnt/powned'-s SECTION/--section=SECTION:
Runs only a portion of the full collection. Can be specified more than once. The full list of sections and subsections is:versionsystem_infokextstartuplaunch_agentsscripting_additionsstartup_itemslogin_itemsapplicationsapplicationsinstall_historyquarantinesdownloadsdownloadsemail_downloadsold_email_downloadschromehistoryarchived_historycookieslogin_datatop_sitesweb_datadatabaseslocal_storagepreferencesfirefoxcookiesdownloadsformhistoryhistorysignonspermissionsaddonsextensioncontent_prefshealth_reportwebapps_storejson_filessafaridownloadshistoryextensionsdatabaseslocalstorageextension_filesaccountssystem_adminssystem_userssocial_accountsrecent_itemsmailfull_hash$sudoosxcollector.py-s'startup'-s'downloads'-c/--collect-cookies:
Collect cookies' value.
By default OSXCollector does not dump the value of a cookie, as it may contain sensitive information (e.g. session id).-l/--collect-local-storage:
Collect the values stored in web browsers' local storage.
By default OSXCollector does not dump the values as they may contain sensitive information.-d/--debug:
Enables verbose output and python breakpoints. If something is wrong with OSXCollector, try this.$sudoosxcollector.py-dDetails of CollectionThe collector outputs a.tar.gzcontaining all the collected artifacts. The archive contains a JSON file with the majority of information.  Additionally, a set of useful logs from the target system logs are included.Common KeysEvery RecordEach line of the JSON file records 1piece of information.  There are some common keys that appear in every JSON record:osxcollector_incident_id: A unique ID shared by every record.osxcollector_section: Thesectionor type of data this record holds.osxcollector_subsection: Thesubsectionor more detailed descriptor of the type of data this record holds.File RecordsFor records representing files there are a bunch of useful keys:atime: The file accessed time.ctime: The file creation time.mtime: The file modified time.file_path: The absolute path to the file.md5: MD5 hash of the file contents.sha1: SHA1 hash of the file contents.sha2: SHA2 hash of the file contents.For records representing downloaded files:xattr-wherefrom: A list containing the source and referrer URLs for the downloaded file.xattr-quarantines: A string describing which application downloaded the file.SQLite RecordsFor records representing a row of a SQLite database:osxcollector_table_name: The table name the row comes from.osxcollector_db_path: The absolute path to the SQLite file.For records that represent data associated with a specific user:osxcollector_username: The name of the userTimestampsOSXCollector attempts to convert timestamps to human readable date/time strings in the formatYYYY-mm-dd hh:MM:ss. It uses heuristics to automatically identify various timestamps:seconds since epochmilliseconds since epochseconds since 2001-01-01seconds since 1601-01-01SectionsversionsectionThe current version of OSXCollector.system_infosectionCollects basic information about the system:system namenode namereleaseversionmachinekextsectionCollects the Kernel extensions from:/System/Library/Extensions/Library/ExtensionsstartupsectionCollects information about theLaunchAgents,
LaunchDaemons, ScriptingAdditions,StartupItemsand other login items from:/System/Library/LaunchAgents/System/Library/LaunchDaemons/Library/LaunchAgents~/Library/LaunchAgents/Library/LaunchDaemons/System/Library/ScriptingAdditions/Library/ScriptingAdditions/System/Library/StartupItems/Library/StartupItems~/Library/Preferences/com.apple.loginitems.plistMore information about the Max OS X startup can be found here:http://www.malicious-streams.com/article/Mac_OSX_Startup.pdfapplicationssectionHashes installed applications and gathers install history from:/Applications~/Applications/Library/Receipts/InstallHistory.plistquarantinessectionQuarantines are basically the info necessary to show the 'Are you sure you wanna run this?' when a user is trying to open a file downloaded from the Internet.
For some more details, checkout the Apple Support explanation of Quarantines:http://support.apple.com/kb/HT3662This section collects also information from XProtect hash-based malware check
for quarantines files. The plist is at:/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/XProtect.plistXProtect also add minimum versions for Internet Plugins. That plist is at:/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/XProtect.meta.plistdownloadssectionHashes all users' downloaded files from:~/Downloads~/Library/Mail Downloads~/Library/Containers/com.apple.mail/Data/Library/Mail DownloadschromesectionCollects following information from Google Chrome web browser:HistoryArchived HistoryCookiesExtensionsLogin DataTop SitesWeb DataThis data is extracted from~/Library/Application Support/Google/Chrome/DefaultfirefoxsectionCollects information from the different SQLite databases in a Firefox profile:CookiesDownloadsForm HistoryHistorySignonsPermissionsAddonsExtensionsContent PreferencesHealth ReportWebapps StoreThis information is extracted from~/Library/Application Support/Firefox/ProfilesFor more details about Firefox profile folder seehttp://kb.mozillazine.org/Profile_folder_-_FirefoxsafarisectionCollects information from the different plists and SQLite databases in a Safari
profile:DownloadsHistoryExtensionsDatabasesLocal StorageaccountssectionCollects information about users' accounts:system admins:/private/var/db/dslocal/nodes/Default/groups/admin.plistsystem users:/private/var/db/dslocal/nodes/Default/userssocial accounts:~/Library/Accounts/Accounts3.sqliteusers' recent items:~/Library/Preferences/com.apple.recentitems.plistmailsectionHashes files in the mail app directories:~/Library/Mail~/Library/Mail Downloadsfull_hashsectionHashes all the files on disk. All of 'em. This does not run by default. It must be triggered with:$sudoosxcollector.py-sfull_hashBasic Manual AnalysisForensic analysis is a bit of art and a bit of science. Every analyst will see a bit of a different story when reading the output from OSXCollector. That's part of what makes analysis fun.Generally, collection is performed on a target machine because something is hinky: anti-virus found a file it doesn't like, deep packet inspect observed a callout, endpoint monitoring noticed a new startup item. The details of this initial alert - a file path, a timestamp, a hash, a domain, an IP, etc. - that's enough to get going.TimestampsSimply greping a few minutes before and after a timestamp works great:$catINCIDENT32.json|grep'2014-01-01 11:3[2-8]'Browser HistoryIt's in there. A tool likejqcan be very helpful to do some fancy output:$catINCIDENT32.json|grep'2014-01-01 11:3[2-8]'|jq'select(has("url"))|.url'A Single User$catINCIDENT32.json|jq'select(.osxcollector_username=="ivanlei")|.'Automated AnalysisTheOSXCollector Output Filters projectcontains filters that process and transform the output of OSXCollector. The goal of filters is to make it easy to analyze OSXCollector output.Development TipsThe functionality of OSXCollector is stored in a single file:osxcollector.py. The collector should run on a naked install of OS X without any additional packages or dependencies.Ensure that all of the OSXCollector tests pass before editing the source code. You can run the tests using:make testAfter making changes to the source code, runmake testagain to verify that your changes did not break any of the tests.LicenseThis work is licensed under the GNU General Public License and a derivation ofhttps://github.com/jipegit/OSXAuditorBlog postOSXCollector: Forensic Collection and Automated Analysis for OS Xby Ivan LeichtlingPresentationsOSXCollector: Automated forensic evidence collection & analysis for OS Xby Kuba Sendor @ BruCON 0x07Squashing Rotten Apples: Automated forensics & analysis for Mac OS X with OSXCollectorby Kuba Sendor @ BSides Manchester 2015OSXCollectorby Ivan Leichtling @ Macbrained's August Meet-UpOSXCollector - Automated Forensic Evidence Collection & Analysis for OS Xby Ivan Leichtling @ OpenNSMOSXCollector - Automated Forensic Evidence Collection & Analysis for OS Xby Ivan Leichtling @ Duo Tech Talk (blog post)External PresentationsOSX Archaeology: Becoming Indiana Jones with OSXCollector and Strataby Chris Henderson & Justin Larson @ SAINTCon 2015ResourcesWant to learn more about OS X forensics?Sarah Edward's mac4n6.com- The best presentations on Mac forensics.A couple of other interesting tools:KnockKnock- KnockKnock is a command line python script that displays persistent OS X binaries that are set to execute automatically at each boot.Grr- Google Rapid Response: remote live forensics for incident responseosquery- SQL powered operating system instrumentation, monitoring, and analytics