Package: assemblyline-v4-service
Assemblyline 4 - Service BaseThis repository provides the base service functionality for Assemblyline 4 services.Creating a new Assemblyline serviceService file structureAn Assemblyline service has the following file structure:assemblyline-service-<service name>
│
├── Dockerfile
├── <service name>.py
└── service_manifest.ymlThis is overview of what each of these does:Dockerfile─ Build file for the service container, seeDockerfilesection below for more details<service name>.py─ Contains main service codeservice_manifest.yml─ Service definition file, seeService manifestsection below for more detailsService manifestEvery service must have aservice_manifest.ymlfile in its root directory. The manifest file presents essential information about the service to the Assemblyline core system, information the system must have before it can run the service.The diagram below shows all the elements that the manifest file can contain, including a brief description of each.# Name of the servicename:ResultSample# Version of the serviceversion:1# Description of the servicedescription:>ALv4 Result example serviceThis service provides examples of how to:- define your service manifest- use the different section types- use tags- use heuristics to score sections- use the att&ck matrix- use the updater framework- define submission parameters- define service configuration parameters# Regex defining the types of files the service accepts and rejectsaccepts:.*rejects:empty|metadata/.*# At which stage the service should run (one of: FILTER, EXTRACT, CORE, SECONDARY, POST)# NOTE: Stages are executed in the order defined in the liststage:CORE# Which category the service is part of (one of: Antivirus, Dynamic Analysis, External, Extraction, Filtering, Networking, Static Analysis)category:Static Analysis# Does the service require access to the file to perform its task# If set to false, the service will only have access to the file metadata (e.g. Hashes, size, type, ...)file_required:true# Maximum execution time the service has before it's considered to be timed outtimeout:60# Does the service force the caching of results to be disabled# (only use for service that will always provided different results each run)disable_cache:false# is the service enabled by defaultenabled:true# does the service make APIs call to other product not part of the assemblyline infrastructure (e.g. VirusTotal, ...)is_external:false# Number of concurrent services allowed to run at the same timelicence_count:0# service configuration block (dictionary of config variables)# NOTE: The key names can be anything and the value can be of any typesconfig:str_config:value1int_config:1list_config:[1,2,3,4]bool_config:false# submission params block: a list of submission param object that define parameters#                          that the user can change about the service for each of its scans# supported types: bool, int, str, listsubmission_params:-default:""name:passwordtype:strvalue:""-default:falsename:extra_worktype:boolvalue:false# Service heuristic blocks: List of heuristics object that define the different heuristics used in the serviceheuristics:-description:This the first Heuristic for ResultSample service.filetype:pdfheur_id:AL_RESULTSAMPLE_1name:Masks has PDFscore:100attack_id:T1001-description:This is second Heuristic for ResultSample service.filetype:exeheur_id:AL_RESULTSAMPLE_2name:Drops an exescore:1000-description:This is third Heuristic for ResultSample service.filetype:exeheur_id:AL_RESULTSAMPLE_3name:Extraction informationscore:0# Docker configuration block which defines:#  - the name of the docker container that will be created#  - cpu and ram allocation by the containerdocker_config:image:cccs/assemblyline-service-resultsample:latestcpu_cores:1.0ram_mb_min:128ram_mb:256# Update configuration blockupdate_config:# list of source object from where to fetch files for update and what will be the name of those files on disksources:-uri:https://file-examples.com/wp-content/uploads/2017/02/zip_2MB.zipname:sample_2mb_file-uri:https://file-examples.com/wp-content/uploads/2017/02/zip_5MB.zipname:sample_5mb_file# intervale in seconds at which the updater runsupdate_interval_seconds:300# Should the downloaded files be used to create signatures in the systemgenerates_signatures:falseDockerfileA Dockerfile is required to build the service container that will be executed in the system.The following items must be set for all services:All services must be based on thecccs/assemblyline-v4-service-base:latestimageAn environment variable must be set for the service pathInstall any service requirementsCopy the service code into/opt/al/al_service/FROMcccs/assemblyline-v4-service-base:latest# Set the service pathENVSERVICE_PATHresult_sample.ResultSample# By default, the base service container as the assemblyline user as the running user#  switch to root to perform installation of dependanciesUSERroot# See that we all these operations in one line to reduce#  the number of container layers and size of the containerRUNapt-getupdate&&apt-getinstall-ymy_debian_apt_dependency_package&&rm-rf/var/lib/apt/lists/*RUNpipinstall--no-cache-dir--usermy_pip_dependency&&rm-rf~/.cache/pip# Change to the assemblyline user to make sure your service does not run as rootUSERassemblyline# Copy the service code in the service directoryWORKDIR/opt/al_serviceCOPYassemblyline_result_sample_service.Testing an Assemblyline serviceTo test an Assemblyline service in standalone mode, therun_service_once.pyscript can be used to run a single task through the service for testing. That script does not require that you have a working version of Assemblyline installed, all you need are the Assemblyline python libraries.Setting up dev environmentNOTE:The following environment setup has only been tested on Ubuntu 20.04.Install required packagessudo apt-get install build-essential libffi-dev python3.7 python3.7-dev python3-pip automake autoconf libtoolInstall Assemblyline v4 service packagepip install --no-cache-dir --user assemblyline-v4-serviceAdd your service development directory path (ie./home/ubuntu/assemblyline-v4-service) to the PYTHONPATH environment variableUsing therun_service_once.pyscriptStepsEnsure the current working directory is the root of the service directory of the service to be runcdassemblyline-service-<servicename>From a terminal, run therun_service_oncescript, where<service path>is the path to the service module and<file path>is the path of the file to be processedpython3.11-massemblyline_v4_service.dev.run_service_once<servicepath><filepath>The output of the service (result.jsonand extracted/supplementary files) will be located in a directory where the input file is locatedExample of running the ResultSample serviceChange working directory to root of the service:cdassemblyline_result_sample_serviceFrom a terminal, run therun_service_oncescriptpython3.11-massemblyline_v4_service.dev.run_service_onceassemblyline_result_sample_service.result_sample.ResultSample/home/ubuntu/testfile.docTheresults.jsonand any extracted/supplementary files will be outputted to/home/ubuntu/testfile_resultsample