Package: naspi
simple-nas-piIntroductionSimple implementation of a NAS on raspberry PI with disk redundancy + S3 glacier backup.Advantagessimplicity: use only simple tools like rsyncreliability: local replication using rsync, per directory. AWS S3 backup for creating off site secured archivescost: cheap hardware, s3 glacier deep archive cost is ~ 0.3 $ a month per 100GBeasy maintenance: use commodity hardware : raspberry Pi, USB3 hard drives. Can be easily replaced in case of failureAssociate with tools you like based on your needs :Plex media server, Nextcloud, Samba shareetc...ArchitectureUsageModesThe different mode of the Cli are :naspi -c /path/to/conf.json-m init_configinit a configuration filenaspi -c /path/to/conf.json-m synclocalsync local folders based on local folder configurationnaspi -c /path/to/conf.json-m syncs3sync local folders to s3 glacier deep archivenaspi -c /path/to/conf.json-m analyzegive local and s3 replication statusnaspi -c /path/to/conf.json-m systemreturn system information cpu, ram, tempnaspi -c /path/to/conf.json-m backupbackup specific files or folders as set in the backup section of the config filenaspi -c /path/to/conf.json-m osbackupbackup the entire sd card to an .img.gz archive fileStatus fileEach run of the CLI updates a status file you will find in the tool working dir. It gives information on the syncronization status, server metrics, disks health and usage etc..
These files are sent to AWS on a regular basis, so mail alerts can be triggered in case of an issue. Emails alerts are also sent in case files are not received, probably meaning the NAS is unreachable.Example status file :{"disks":{"disk-list":[{"name":"/disks/disk2","occupied_%":"13% ","present":true},{"name":"/disks/disk1","occupied_%":"13% ","present":true},{"name":" /","occupied_%":" 46%","present":true}],"all_disks_ok":true,"last_run":"2021-02-10 23:50:01"},"local_sync":{"success":true,"files_source":101255,"files_dest":101257,"files_delta":-2,"locked":false,"last_started":"2021-02-10 23:14:17","last_run":"2021-02-10 23:21:02"},"s3_sync":{"success":true,"files_source":23105,"files_dest":23105,"files_delta":0,"locked":false,"last_started":"2021-02-10 17:32:35","last_run":"2021-02-10 17:33:25"},"server":{"cpu_%":" 1,4 ","ram_Mo":" 508 ","temp_c":"50.6'C","last_run":"2021-02-10 23:50:02"}}InstallationprerequisitesA raspberry pi4 (should work with pi3) with pi OS installed2 USB3 disks (might require power supply as the Pi can't power up 2 * 2.5 disks)Install disks and configure the mount points in /etc/fstabAn AWS account with admin accessInstallationInstall naspi from Pypi :pip3installnaspiConfigure NaspiInitialize a new config filenaspi-c./naspi_config.json-minit_configConfigure the toolvi./naspi_config.jsonInitially the config file is :{"disks_list":[],"folder_to_sync_locally":[],"folders_to_sync_s3":[],"naspi_configuration":{"working_dir":"","NUMBER_DAYS_RETENTION":7,"MIN_DELAY_BETWEEN_SYNCS_SECONDS":14400,"backup":{"files_to_backup":[],"backup_location":"","os_backup_location":""}}}Set the"working_dir"to a directory to store the naspi files (logs, config, status files)Set the"disks_list": mount points of the disks storing the data so they can be monitored"disks_list":["/disks/disk1","/disks/disk2"]Set"folder_to_sync_locally"following the examples below. "delete" option means the deletion are replicated as well."folder_to_sync_locally":[{"source_folder":"/disks/disk1/media/photos/","dest_folder":"/disks/disk2/media/photos/","delete":false},{"source_folder":"/disks/disk1/media/download/","dest_folder":"/disks/disk2/media/download/","delete":true}]Set "folders_to_sync_s3".delete option not implemented yet"folders_to_sync_s3":[{"source_folder":"/disks/disk1/media/photos/","dest_folder":"s3://<bucket-name>/photos","exclude":["folder-to-exclude"],"delete":false},{"source_folder":"/disks/disk1/media/download","dest_folder":"s3://<bucket-name>/download","delete":false}]Set "naspi_configuration" block with files you need to backup"naspi_configuration":{"working_dir":"/home/pi/naspi","NUMBER_DAYS_RETENTION":7,"MIN_DELAY_BETWEEN_SYNCS_SECONDS":14400,"backup":{"files_to_backup":["/etc/fstab","/home/pi","/etc/samba/smb.conf"],"backup_location":"/disks/disk1/backups/","os_backup_location":"/disks/disk1/osbackups/"}}Set the crons : naspi CLI will be invoked based on a cron schedule.
Export the path with your local user to make the naspi command availablecrontab-e1101***exportPATH=/home/pi/.local/bin:$PATH&&naspi-c/home/pi/naspi/naspi_config.json-mbackup3217***exportPATH=/home/pi/.local/bin:$PATH&&naspi-c/home/pi/naspi/naspi_config.json-msyncs306****exportPATH=/home/pi/.local/bin:$PATH&&naspi-c/home/pi/naspi/naspi_config.json-msynclocal
*/10****exportPATH=/home/pi/.local/bin:$PATH&&naspi-c/home/pi/naspi/naspi_config.json-msystem113**2exportPATH=/home/pi/.local/bin:$PATH&&naspi-c/home/pi/nas_monitor/naspi_config.json-mosbackupDeploy resources in AWS accountSeveral resources are deployed in AWS : S3 bucket, user, monitoring lambda functions, SNS topic for email notifications.In your AWS account, with administrator access : go tocloudformationserviceCreate a new stack using the template aws/deply-naspi.ymlParameters of the stack are :NaspiBucketName(REQUIRED): Bucket Name to save the content backed up from the NASEmailForReceivingAlerts(REQUIRED) : Email address to receive the NAS alertsMonitoringSchedule : This defines the Schedule at which to trigger the Naspi monitoring function. Default: cron(0 15 ? * * *)Generate access keysAWS Access keys will give the NAS access to the AWS account (S3 bucket)In your AWS account, with administrator access : go toIAMserviceFind the userNasPiUserGo to security credentials, generate an access key / secret key pairConfigure AWS CLIInsert the access key / secret key obtained before in˜/.aws/credentialsfile :[default]aws_access_key_id=AKIAJXXXXXXXXXXXaws_secret_access_key=XXXXXXXXXXXXXXXXXXXXXXXX