Adding Security to Your Code
By: Brandon Walker
Tags: Programming, Security, Automation, Data Science, Continuous Integration

Adding Security to Your Code

A phrase I often hear is “security is everyone’s responsibility” but I notice that data scientists are frequently so focused on the vast number of skills that they need to know, that security goes ignored. Besides having many responsibilities, I believe that security seems daunting and appears to require lots of software engineering skill. In reality, it’s fairly easy to implement the lowest level of security into your software. I’d recommend following Charles Nwatu’s (a leader in security at Netflix and formerly StitchFix) principle “do less better.” To me, this means successfully implementing low level security is better than failing to implement high level security. This article is aimed at data scientists (or python users) and only assumes you have some rudimentary knowledge of the command line.

Types of Security Tools

There are two broad types of security tools, static and dynamic. Dynamic security tools run against running software to uncover threats, while static tools run against source code files to find problems. In this article we will be adding two static security tools. The first tool will check to make sure you are not adding keys, secrets, or passwords to your repository, preventing you from exposing that private information. The second tool will help you check your software dependencies for security threats.

Git Secrets

Git Secrets is the tool we will use to monitor our repositories for information we don’t want to be public. It is available and easy to install for your operating system by consulting its repo here. Once you have installed it you can add things we want to look out for with the following commands: git secrets --add 'your-regular-expression' to block patterns or git secrets --add --literal 'your-literal-string' to block specific strings. As an example, I will run git secrets --add 'password ?=+ ?[A-Za-z0-9]+ which will block things like the following:

password = anyLengthPassword

password=passwordWithNoSpacesNextToEqualSign

password==doubleEqualsBlockedToo

But won’t block these:

password=””

password=

This is great since it won’t warn us against pushing code that has the passwords properly removed. Now I run git secrets --scan -r to scan all the documents for the patterns I added. It found one password that I put on the first line of the README. Here’s what Git Secrets reported to me:

README.md:1:password = password12345



[ERROR] Matched one or more prohibited patterns



Possible mitigations:

- Mark false positives as allowed using: git config --add secrets.allowed ...

- Mark false positives as allowed by adding regular expressions to .gitallowed at repository's root directory

- List your configured patterns: git config --get-all secrets.patterns

- List your configured allowed patterns: git config --get-all secrets.allowed

- List your configured allowed patterns in .gitallowed at repository's root directory

- Use --no-verify if this is a one-time false positive

I set these Git Secrets configurations for this specific repository, but you can set some global configurations to prevent you from pushing any secrets in any repo. Do this with git secrets --add --global 'pattern or string here' .

Virtual Environments

In a previous article I argued that creating a virtual environment should be one of the first steps you take on almost any project. A virtual environment allows you to isolate software for different projects, so if you need pandas version 0.25.3 on one project and pandas version 1.1.2 on another there will be no problems.

If you are indeed using python you will most likely be using conda or virtualenv. To create a virtual environment with conda run conda create --name 'env-name' python='python-version' and to create a virtual environment with virtualenv use virtualenv 'env-name' . You can store all of your python packages into a file usually called requirements.txt, which is useful for sharing with others if you want to work collaboratively on the same code in the same environment. This is all my requirements.txt file has in it:

numpy==1.16.0

To install all the packages in your requirements.txt file run pip install -r requirements.txt and to generate a requirements.txt file run pip freeze > requirements.txt .

BOM

According to a 2020 DevSecOps Community Survey, 28% of respondents said they had at least one security breach related to an opensource tool. So how do you uncover these security threats before they become problems? By using a Bill of Materials (BOM), which will describe all the software dependencies and libraries that your project uses. This BOM can then be compared against a public record of discovered security threats which will bring problems to your attention. If you’re already using a virtual environment, your requirements.txt file can serve as a BOM for your python project. Now let’s install safety, which will scan your active virtual environment or a requirements.txt file and compare it against a database of known vulnerability database. To do this run pip install safety . As a warning, this database is only updated once a month, to get access to the very latest security threats, you’d have to pay PyUp, the company behind safety. But remember, “do less, better.”

Now that you have it installed you can run safety check on an active environment or run safety check -r requirements.txt to check a requirements.txt file. I’ve done this against my requirements file that only has numpy version 1.16.0 in it. Here’s the output:

Safety Check

I am now alerted to a security threat in numpy version 1.16.0 and I now know to upgrade to a different version.

Running These Tools in One Script

We can put both of these commands into one shell script that can be executed when you want. I opened a new text document to put these in with nano security.sh . In this file I wrote:

#!/bin/bash

# This script runs a security check

git secrets --scan -r

safety check -r requirements.txt

The first line tells us where bash is located (yours may be different). There are lots of interpreters out there and I want bash (which is one of the more common ones) to execute this script. The second line is a conventional comment that tells us what the script does. As you learn about new tools you can add them to this one script, which will make executing security checks much simpler.

Continuous Integration

What’s even better than running this script occasionally is beginning to adopt a continuous integration pipeline. Continuous integration is the process of working on small batches of code at a time and integrating your code with a main branch frequently. When you do integrate your code you can automatically run tests for code quality along with these checks for security. If you do this, you will know about problems sooner and consequentially mitigate them faster.

To make use of continuous integration you may want to start by selecting a tool for your team to use like CircleCI, TravisCI, or Jenkins. These tools can be hooked into your repo in GitHub, Bitbucket, or GitLab and begin execution when there is a change there. Below is an example of a Jenkinsfile that you could use to automate the above as well as any python tests you may write in the tests.py file. This would be executed any time you or any of your teammates makes a push to your repository.

pipeline {

agent any

stages {

stage('SecTest') {

steps {

sh ' git secrets --scan -r '

sh ' safety check -r requirements.txt '

}

}

stage('AppTest') {

steps {

sh 'python.exe tests.py'

}

}

}

}

You could even add additional steps that deploy any machine learning models or functions that you create to this pipeline. In my opinion code with high security and quality is easier to develop once you have adopted a CI pipeline. Hopefully, this gets you started on improving the security in your code!