Package: duetector
duetector🔍: Data Usage Extensible detector(eBPF Support)English|中文Introductionduetector is one of the components in the DataUCON project, which is designed to provide support for data usage control.Intro DataUCON.duetector🔍 is an extensible data usage control detector that provides support for data usage control by probing for data usage behavior in the Linux kernel(based on eBPF).🐛🐞🧪 The project is under heavy development, looking forward to any bug reports, feature requests, pull requests!In theABAUC control model, duetector can be used as a PIP (Policy Information Point) to obtain data usage behavior, so as to provide information about data usage behavior for PDP (Policy Decision Point). Provide information on data usage behavior to PDP (Policy Decision Point).Try simple user case:Simplest Open Count.Join ourslack channel.Table of ContentsFeaturesInstallationQuick StartAPIMaintainersContributorsHow to contributeLicenseFeaturePlug-in system support, seeexamplesfor more detailsCustomTracerandTracerManagerCustomFiltersandFilterManagerCustomCollectorandCollectorManagerCustomAnalyzerandAnalyzerManagerConfiguration ManagementConfiguration using a single configuration fileGenerate Plugin ConfigurationSupport for dynamically loading configurationsTracerSupporteBPF-based tracerShell command tracerSubprocess tracerFilterSupportPattern matching, based on regular expressionsData Collection and AnalysisAnalyzerSupport SQL databaseCollectorSupport SQL database andOpenTelemetry(Experimental)User InterfaceCLI ToolsPIP ServiceControl PanelEnhancementsRunCcontainers identificationThe eBPF program requires kernel support, seeKernel SupportInstallationThe code is distributed via Pypi, and you can install it with the following commandpipinstallduetectorCurrently, the code relies onBCCfor on-the-fly compilation of eBPF code, we recommendinstalling the latest BCC compilerOr use the Docker image that we provide, which usesJupyterLabas theexampleuser application, or you can modify theDockerfileandstartup scriptto customize the user application.dockerpulldataucon/duetector:latestPre-releases will not be updated tolatest, you can specify the tag to pull, e.g.v0.0.1adockerpulldataucon/duetector:v0.0.1aFor more details on running with docker images seehereQuick startMore documentation and examples can be foundhere.Start detectorStart monitor using the command line, since bcc requires root privileges, we use thesudocommand, which will start all probes and collect the probes into theduetector-dbcollector.sqlite3file in the current directorysudoduectlstartPressCRTL+Cto exit monitoring and you will see a summary output on the screen{'DBCollector': {'OpenTracer': {'count': 31, 'first at': 249920233249912, 'last': Tracking(tracer='OpenTracer', pid=641616, uid=1000, gid= 1000, comm='node', cwd=None, fname='SOME-FILE', timestamp=249923762308577, extended={})}}}EnableDEBUGlogsudoDUETECTOR_LOG_LEVEL=DEBUGduectlstartAt startup, the configuration file will be automatically generated at~/.config/duetector, and you can specify the configuration file to use with--config.sudoduectlstart--config<config-file-path>Configuration using environment variables is also supported:Usage:duectlstart[OPTIONS]StartAbccmonitorandwaitforKeyboardInterrupt

Options:...--load_envBOOLEANWeatherloadenvvariables,Prefix:DUETECTOR_,Separator:__,e.g.DUETECTOR_config__ameansconfig.a,default:True...When using a plugin, the default configuration file will not contain the plugin's configuration, use the dynamically-generated configuration directive to generate a configuration file with the plugin's configuration, this directive also supports merging existing configuration files and environment variables.duectlgenerate-dynamic-config--helpUsegenerate-configto restore the default state in case of configuration file errors.duectlgenerate-configGoing a step further, running in the background you can use theduectl-daemon startcommand, which will run a daemon in the background, which you can stop usingduectl-daemon stopUseduectl-daemon --helpfor more details:Usage:duectl-daemon[OPTIONS]COMMAND[ARGS]...

Options:--helpShowthismessageandexit.

Commands:startStartabackgroundprocessofcommand`duectlstart`.statusShowstatusofprocess.stopStoptheprocess.Analyzing with analyzerWe provide anAnalyzerthat can query the data in storage, try it inuser caseUsing duetector serverWe provide a Duetector Server as an external PIP service and control interfaceA Duetector Server can be started usingduectl-serverand will listen on0.0.0.0:8120by default, you can modify it using--hostand--port.$duectl-serverstart--help
Usage:duectl-serverstart[OPTIONS]Startduetectorserver

Options:--configTEXTConfigfilepath,default:``~/.config/duetector/config.toml``.--load_envBOOLEANWeatherloadenvvariables,Prefix:``DUETECTOR_``,Separator:``__``,e.g.``DUETECTOR_config__a``means``config.a``,default:True--workdirTEXTWorkingdirectory,default:``.``.--hostTEXTHosttolisten,default:``0.0.0.0``.--portINTEGERPorttolisten,default:``8120``.--workersINTEGERNumberofworkerprocesses,default:``1``.--helpShowthismessageandexit.After the service has started, visithttp://{ip}:{port}/docsto see the API documentation.Similarly, usingduectl-server-daemon startyou can run a Duetector Server in the background, and you can stop it usingduectl-server-daemon stop$duectl-server-daemon
Usage:duectl-server-daemon[OPTIONS]COMMAND[ARGS]...

Options:--helpShowthismessageandexit.

Commands:startStartabackgroundprocessofcommand``duectl-serverstart``.statusShowstatusofprocess.stopStoptheprocess.API documentationSeedocs of duetectorMaintainersThis project is initiated byInstitute of Data Security, Harbin Institute of Technology (Shen Zhen), if you are interested in this project andDataUCONproject and willing to work together to improve it, welcome to join our open source community.Contributorswunder957💻MayDown💻tsdsnk📖zhemulin📖Mortal📖mingzhedream📖How to contributeStarting with thegood first issueand reading ourcontributing guidelines.Learn about the designing and architecture of this project here:docs/design.LicenseThis project uses Apache-2.0 license, please refer toLICENSE.