Package: dgaintel
DGA IntelUsing deep learning to detect DGA domains.OverviewThe DGAIntel Python module allows you to utilize a powerful CNN-LSTM model to predict whether a given domain name was generated by a domain generation algorithm (DGA) or corresponds to a genuine domain. The prediction features are also accesible throughthis website, but this package allows for direct integration into your workflow.RequirementsDGAIntel is designed for use with Python 3. It has only two requirements:- TensorFlow 2.x
- NumpyInstallationTo download dgaintel, simply use Pypi via pip.$pipinstalldgaintelAlternatively, you could install from source.$gitclonehttps://github.com/sudo-rushil/dgaintel
$cddgaintel
$pythonsetup.pyinstallVerify your installation by running>>>importdgaintel>>>dgaintel.get_prediction('microsoft.com')'microsoft.com is genuine with probability 0.00050'ExamplesPredict DGAThis is simple way of determining whether any given domain, such as'microsoft.com'is DGA or not, mainly intended for cyber security analysts.fromdgaintelimportget_predictionget_prediction('microsoft.com')'microsoft.com is genuine with probability 0.00050'Predict DGA probabilityThis allows for getting the probability, or probabilities, that a domain or list of domains is DGA or not, which is more useful to data scientists.fromdgaintelimportget_prob# For single domainprob=get_prob('microsoft.com')print(prob)# For multiple domainsprobs=get_prob(['microsoft.com','wikipedia.com','vlurgpeddygdy.com'])print(probs)# To get just the scoresraw_probs=list(get_prob(['microsoft.com','wikipedia.com','vlurgpeddygdy.com'],raw=True))print(raw_probs)0.00050[('microsoft.com', 0.00050), ('wikipedia.com', 0.00033), ('vlurgpeddygdy.com', 0.97601)][0.00050845, 0.00033092, 0.00144754]Predict by fileThis is for inputing a file containing a list of domains to get predictions on all of them at once, which is helpful for data analysts.Say you have a domain filedomains.txt.microsoft.com
wikipedia.com
vlurgpeddygdy.comThen, you can run the following code in the same directory.fromdgaintelimportget_prediction# Print to consoleget_prediction('domains.txt')# Write to fileget_prediction('domains.txt',to_file='domain_predictions.txt')microsoft.com is genuine with probability 0.00050wikipedia.com is genuine with probability 0.00033vlurgpeddygdy.com is DGA with probability 0.97601If you read the new filedomain_predictions.txt, you will see the following.microsoft.com is genuine with probability 0.0005084535223431885
wikipedia.com is genuine with probability 0.00033092446392402053
vlurgpeddygdy.com is DGA with probability 0.9760094285011292Prediction analysisThis is an example function that integrates dgaintel withwhoisfor performing basic prediction analysis, which is important for cyber security investigators.fromdgaintelimportget_probfromwhoisimportquerydefanalyze(domain,out=True):prob=get_prob(domain)whois=query(domain)dga=Falseifprob>=0.5:dga=Truedomain_analysis={'domain_name':domain,'dga':dga,'registrar':whois.registrarifwhoiselseNone,'creation date':whois.creation_dateifwhoiselseNone,'expiration date':whois.expiration_dateifwhoiselseNone}ifout:print()forkey,valindomain_analysis.items():print('{}:{}'.format(key,val))print()returnNonereturndomain_analysisanalyze('microsoft.com')# Get analysis dictionary in python itselfanalysis=analyze('microsoft.com',out=False)name: microsoft.comdga: Falseregistrar: MarkMonitor Inc.creation date: 1991-05-02 04:00:00expiration date: 2021-05-03 04:00:00Predictions with WhitelistingThis example shows how the class interface to DGAIntel allows for certain TLDs to be whitelisted, preventing them from raising errors in a given ecosystem.fromdgaintelimportIntelintel=Intel(['cloud.com'])print(intel.get_prob(['www.cloud.com','dfsadkcda.cloud.com','www.cloud.org','www.dkfjsdakfj.org']))[('www.cloud.com', 0.0), ('dfsadkcda.cloud.com', 0.0), ('www.cloud.org', 0.00045579672), ('www.dkfjsdakfj.org', 0.99884665)]DocumentationDGAIntel has support for polymorphism; to input domains to run predictions on, you can use a single domain name, a list of domain names, or a text file with line-separated domain names. The text file has the formatmicrosoft.com
wikipedia.com
vlurgpeddygdy.com
...Additionally, the Tensorflow Keras model running in the backend supports input batching, meaning there is a significant increase in speed for running predictions on lists or files rather than individual domains. This was tested in Jupyter.fromdgaintelimportget_prob# List of 10 domain namesl=['microsoft.com','squarespace.com','hsfkjdshfjasdhfk.com','fdkhakshfda.com','foilfencersarebad.com','foilfencersarebad.com','foilfencersarebad.com','discojjfdsf.com','fasddafhkj.com','wikipedai.com']# One domain%%timeitget_prob(l[0])286 ms ± 4.99 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)# Ten domains%%timeitget_prob(l)290 ms ± 7.23 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)# Hundred domains%%timeitget_prob(l*10)333 ms ± 4.71 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)# Thousand domains%%timeitget_prob(l*100)584 ms ± 14.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)This demonstrates that increasing the number of domain names one runs the prediction by 1000x only increases the inference time by less than 2x. Therefore, this model is easily adaptable to large-scale predictions.APITheget_predictionfunction will either print the predictions or write them to a user-specified file.fromdgaintelimportget_predictionget_prediction('microsoft.com')get_prediction(['microsoft.com','wikipedia.com','vlurgpeddygdy.com'])get_prediction('domains.txt')get_prediction('domains.txt',to_file='domain_predictions.txt')Theget_probfunction will perform the inference and provide the prediction floats. It is helpful if you want to use the prediction scores directly in your workflow.fromdgaintelimportget_probget_prob('microsoft.com')# 0.00050851get_prob(['microsoft.com','wikipedia.com','vlurgpeddygdy.com'])# [('microsoft.com', 0.00050), ('wikipedia.com', 0.00033), ('vlurgpeddygdy.com', 0.0.97601)]get_prob('domains.txt')# [('microsoft.com', 0.00050), ('wikipedia.com', 0.00033), ('vlurgpeddygdy.com', 0.97601)]get_prob(['microsoft.com','wikipedia.com','google.com'],raw=True)# array([0.00050, 0.00033, 0.0.97601], dtype=float32)TheIntelinterface allows DGAIntel to avoid checking certain domains with known TLDs to ensure enterprise functions are not compromised.fromdgaintelimportIntelintel=Intel(['microsoft.com'])intel.get_prob('microsoft.com')# 0.0intel.get_prob(['microsoft.com','wikipedia.com','vlurgpeddygdy.com'])# [('microsoft.com', 0.0), ('wikipedia.com', 0.00033), ('vlurgpeddygdy.com', 0.0.97601)]intel.get_prob('domains.txt')# [('microsoft.com', 0.0), ('wikipedia.com', 0.00033), ('vlurgpeddygdy.com', 0.97601)]intel.get_prob(['microsoft.com','wikipedia.com','google.com'],raw=True)# array([0.0, 0.00033, 0.0.97601], dtype=float32)