Package: aexpy
AexPy/eɪkspaɪ/isApiEXplorer inPYthon for detecting API breaking changes in Python packages. ExploreAPIs of AexPy releases, andAPIs of main branchbyAexPyitself.[!NOTE]
AexPy is the prototype implementation of the conference paper "AexPy: Detecting API Breaking Changes in Python Packages" in Proceedings of the 33rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2022), Charlotte, North Carolina, USA, October 31 - November 3, 2022.If you use our approach or results in your work, please cite it according tothe citation file.X. Du and J. Ma, "AexPy: Detecting API Breaking Changes in Python Packages," 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE), 2022, pp. 470-481, doi: 10.1109/ISSRE55969.2022.00052.https://user-images.githubusercontent.com/34736356/182772349-af0a5f20-d009-4daa-b4a9-593922ed66fe.movHow AexPy works?Approach Design & Evaluation are inAexPy's conference paper, see alsotalk&slides.How we implement AexPy?Source Code & Implemetation are inAexPy's repository, see alsodesign (zh-cn).How to use AexPy?Detailed Document & Data are inAexPy's documents, see alsovideoandonline AexPy.graph LR;
    Package-->Version-1;
    Package-->Version-2;
    Version-1-->Preprocessing-1;
    Version-2-->Preprocessing-2;
    Preprocessing-1-->Extraction-1;
    Preprocessing-2-->Extraction-2;
    Extraction-1-->Difference;
    Extraction-2-->Difference;
    Difference-->Evaluation;
    Evaluation-->Breaking-Changes;AexPy also provides a framework to process Python packages, extract APIs, and detect changes, which is designed for easily reusing and customizing. See the following "Advanced Tools" section and the source code for details.Quick StartTake the packagegenerator-oj-problemv0.0.1 and v0.0.2 as an example.Save API descriptions tocache/api1.jsonandcache/api2.jsonOutput report toreport.txt# Install AexPy package and toolpipinstallaexpy# Extract APIs from v0.0.1echogenerator-oj-problem@0.0.1|aexpyextract-api1.json-r# Extract APIs from v0.0.1echogenerator-oj-problem@0.0.2|aexpyextract-api2.json-r# Diff APIs between two versionsaexpydiffapi1.jsonapi2.jsonchanges.jsonView results ononline AexPy.generator-oj-problem@0.0.1DistributionandAPIgenerator-oj-problem@0.0.2DistributionandAPIChangesandReportSee also aboutAPI Level,Call Graph, andInheritance Diagram.FeaturesPreprocessingDownload packages and get source code, or use existing code base.Count package file sizes and lines of code.Read package metadata and detect top modules.ExtractingExtract APIs from Python packages, including modules, classes, functions, attributes.Collect detailed APIs, including parameters, instance attributes.Detect API aliases and build call graphs, inheritance diagrams.Enrich type information for APIs by static type analyzers.DiffingDetect API changes after pairing APIs between two versions.Grade changes by their severities.ReportingGenerate a human-readable report for API change detection results.FrameworkCustomize processors and implementation details.Process Python packages in AexPy's general pipeline with logging and caching.Generate portable data in JSON for API descriptions, changes, and so on.Execute processing and view data by AexPy's command-line, with stdin/stdout supported.InstallWe provide the Python package on PyPI. Use pip to install the package.python-mpipinstall--upgradeaexpy
aexpy--help[!IMPORTANT]
Please ensure your Python interpreter works inUTF-8 mode.We also provide the Docker image to avoid environment errors.dockerpullstardustdl/aexpy:latest
dockerrun--rmstardustdl/aexpy:latest--help# or the image from the main branchdockerpullstardustdl/aexpy:mainUsage[!TIP]
All results produced by AexPy are in JSON format, so you could modify it in any text editor.PreprocessPreprocess a distribution for a package release.AexPy provide four preprocessing modes:-s,--src: (default) Use given distribution information (path to code, package name, modules)-r,--release: download and unpack the package wheel and automatically load from dist-info-w,--wheel: Unpack existing package wheel file and automatically load from dist-info-d,--dist: Automatically load from unpacked wheel, and its dist-infoAexPy will automatically load package name, version, top-level modules, and dependencies from dist-info.There are also options to specify fields in the distribution:-p,--project: Package name and its version, e.g.project@version.-m,--module: (multiple) Top-level module names.-D,--depends: (multiple) Package dependencies.-R,--requirements: Packagerequirements.txtfile path, to load dependencies.-P,--pyversion: Specify Python version for this distribution, supported Python 3.8+.[!TIP]
You could modify the generated distribution file in a text editor to change field values.# download the package wheel and unpack into ./cache# output the distribution file to ./cache/distribution.jsonaexpypreprocess-r-pgenerator-oj-problem@0.0.1./cache./cache/distribution.json# or output the distribution file to stdoutaexpypreprocess-r-pgenerator-oj-problem@0.0.1./cache-# use existing wheel fileaexpypreprocess-w./cache/generator_oj_problem-0.0.1-py3-none-any.whl./cache/distribution.json# use existing unpacked wheel directory, auto load metadata from .dist-info directoryaexpypreprocess-d./cache/generator_oj_problem-0.0.1-py3-none-any./cache/distribution.json# use existing source code directory, given the package's name, version, and top-level modulesaexpypreprocess./cache/generator_oj_problem-0.0.1-py3-none-any./cache/distribution.json-pgenerator-oj-problem@0.0.1-mgenerator_oj_problemView results atAexPy Online.ExtractExtract the API description from a distribution.AexPy provide four modes for the input distribution file:-j,--json: (default) The file is the JSON file produced by AexPy (preprocesscommand)-r,--release: The file is a text containing the release ID, e.g.,aexpy@0.1.0-w,--wheel: The file is a wheel, i.e.,.whlfile-s,--src: The file is a ZIP file that contains the package code directoryPlease ensure the directory is at the root of the ZIP archive[!IMPORTANT]About DependenciesAexPy would dynamically import the target module to detect all available APIs. So please ensure all dependencies have been installed in the extraction environment, or specify thedependenciesfield in the distribution, and AexPy will install them into the extraction environment.If thewheelFilefield is valid (i.e. the target file exists), AexPy will firstly try to install the wheel and ignore thedependenciesfield (used when the wheel installation fails).[!TIP]About EnvironmentAexPy usemicromambaas default environment manager. UseAEXPY_ENV_PROVIDERenvironment variable to specifyconda,mamba, ormicromamba.Use flag--no-tempto let AexPy use the current Python environment (as same as AexPy) as the extraction environment (the default behavior of the installed AexPy package).Use flag--tempto let AexPy create a temporary mamba(conda) environment that matches the distribution's pyverion field (the default behavior of our docker image).Use option-e,--envto specify an existing mamba(conda) env name as the extraction environment (will ignore the temp flag).aexpyextract./cache/distribution.json./cache/api.json# or input the distribution file from stdin# (this feature is also supported in other commands)aexpyextract-./cache/api.json# or output the api description file to stdoutaexpyextract./cache/distribution.json-# extract from the target project releaseechoaexpy@0.0.1|aexpyextract-api.json-r# extract from the wheel fileaexpyextract./temp/aexpy-0.1.0.whlapi.json-w# extract from the project source code ZIP archivezip-r-./project|aexpyextract-api.json-s# Use a env named demo-envaexpyextract./cache/distribution.json--edemo-env# Create a temporary envaexpyextract./cache/distribution.json---tempView results atAexPy Online.DiffDiff two API descriptions and detect changes.aexpydiff./cache/api1.json./cache/api2.json./cache/diff.jsonIf you have both stdin for OLD and NEW, please split two API descriptions by a comma,.echo","|cat./api1.json-./api2.json|aexpydiff--./changes.jsonView results atAexPy Online.ReportGenerate report from detect changes.aexpyreport./cache/diff.json./cache/report.jsonView results atAexPy Online.ViewView produced data.aexpyview./cache/distribution1.json
aexpyview./cache/distribution2.json
aexpyview./cache/api1.json
aexpyview./cache/api2.json
aexpyview./cache/diff.json
aexpyview./cache/report.jsonDocker ImageThe docker image keeps the same command-line interface, but always use stdin/stdout for host-container data transferring.echogenerator-oj-problem@0.0.1|dockerrun-iaexpy/aexpyextract-->./api.json

cat./api1.json<(echo",")./api2.json|dockerrun-iaexpy/aexpydiff--->./changes.json[!TIP]
If you want to write processed data to filesystem, not the standard IO, add a volume mapping to/datafor file access.Since the container runs in non-root user, please use root user to allow the container writing to the mounted directory.dockerrun-v$pwd/cache:/data-urootaexpy/aexpyextract/data/distribution.json/data/api.jsonAdvanced ToolsLoggingThe processing may cost time, you can use multiple-vfor verbose logs (which are outputed to stderr).aexpy-vvvview./cache/report.jsonInteractiveAdd-ior--interactto enable interactive mode, every command will create an interactive Python shell after finishing processing. Here are some useful variable you could use in the interactive Python shell.result: The produced data objectcontext: The producing context, useexceptionto access the exception if failing to processaexpy-iview./cache/report.json[!TIP]
Feel free to uselocals()anddir()to explore the interactive environment.PipelineAexPy has four loosely-coupled stages in its pipeline. The adjacent stages transfer data by JSON, defined inmodelsdirectory. You can easily write your own implementation for every stage, and combine your implementation into the pipeline.