sage-deps 0.1.2
Sage Deps
Requires Python: >=3.7

# Sage Deps - A Sage Dependency Analysis Tool

This is a ![SageMath](https://www.sagemath.org/) dependency analysis tool used for understanding Sage's structure. It can be used to detect circular imports, upstream dependencies, and provide various dependency-graph metrics including PageRank.

**Warning:** This package is still in its testing phase, and may have unexpected behaviour.

# Installation

## Dependencies
The package requires ![numpy](https://numpy.org/), ![scipy](https://scipy.org/), ![networkx](https://networkx.org/), and ![scikit-network](https://pypi.org/project/scikit-network/).

## Install from PyPi:
Run `pip install sage-deps` to install as a pypi package. Make sure the script `sdeps.exe` is added to PATH to run the `sdeps` command.

## Manual Install
To install, simply download or clone the repository. Navigate to the root folder (where `pyproject.toml` lives) and run `pip install .`.

## Additional Setup

The package requires access to the ![SageMath repository](https://github.com/sagemath/sage). By default, it assumes you have the repository cloned to the same parent directory as your sage-deps folder. You may modify this by modifying the configuration file, as shown below.


## Getting started

To ensure the package is successfully installed, run `sdeps`. It should output a blank line by default. 

The package comes with some pre-compiled resource files generated by SageMath version 10.6. To update these pre-generated files, run:
```
sdeps -gm -gg
```

To store and use local resource files, use the `-m` and `-g` flags or run:
```
sdeps -set-config modules_src <modules-path>
sdeps -set-config graph_src <graph-path>
``` 

Similarly, to specify a custom SageMath installation path, use the `-s` flag or run:
```
sdeps -set-config sage_path <sage-path>
```

Relative and absolute files are both supported.

## Full options list

```
usage: sdeps [-h] [-s SOURCE_FILE] [-m SOURCE_FILE] [-g SOURCE_FILE] [-o OUTPUT_FILE] [-up SOURCE_CLASS DEPTH]
             [-cc SOURCE_CLASS TIMEOUT] [-cl SIZE] [-gm] [-gi] [-gd] [-gg] [-gdg SOURCE DISTANCE DIRECTION] [-f SOURCE_FILE] [-nf]
             [-view] [-set-config NAME VALUE] [--verbose]

A program top help manage SageMath dependencies.

options:
  -h, --help            show this help message and exit
  -s, --sage-source SOURCE_FILE
                        The source file of Sage.
  -m, --modules-source SOURCE_FILE
                        Specify the modules file.
  -g, --graph-source SOURCE_FILE
                        Specify the graph file.
  -o, --output-file OUTPUT_FILE
                        Specify the file to dump the output.
  -up SOURCE_CLASS DEPTH
                        Generates a breadth-first dependency tree rooted at SOURCE_CLASS, up to a given depth.
  -cc SOURCE_CLASS TIMEOUT
                        Finds cycles starting at SOURCE_CLASS with a timeout of TIMEOUT seconds.
  -cl SIZE              Finds cliques of size SIZE.
  -gm, --generate-modules
                        Generate a modules file. Will output to default location or `--modules-source`.
  -gi, --generate-imports
                        Generate an imports file.
  -gd, --generate-dependencies
                        Generate a dependencies file.
  -gg, --generate-graph
                        Generate an graph file. Will output to default location or `--graph-source`.
  -gdg, --generate-dependency-graph SOURCE DISTANCE DIRECTION
                        Generate a dependency graph rooted at a SOURCE node.
  -f, --ff SOURCE_FILE  Load a custom filter. If not, a default filter is used.
  -nf                   Disables the filter.
  -view, --view         Run a cytoscape.js instance. Specify graph source using `--graph-source`.
  -set-config NAME VALUE
                        Updates the configuration file.
  --verbose             Enable verbose output.
```

Note: Currently, verbose output doesn't do anything.

# Example Usage

## Command line examples

We can use the `-up` flag to compute all upwards dependencies of a particular class in SageMath.
```
% sdeps -up sage.rings.polynomial.polynomial\_element.Polynomial 1
{0: ['sage.rings.polynomial.polynomial_element.Polynomial'], 1: ['sage.rings.integer.Integer', 'sage.rings.rational_field.RationalField', 'sage.rings.integer_ring.IntegerRing_class', 'sage.rings.padics.padic_generic.pAdicGeneric', 'sage.rings.ideal.Ideal_generic', 'sage.rings.polynomial.polynomial_ring.PolynomialRing_generic', 'sage.rings.polynomial.multi_polynomial.MPolynomial', 'sage.rings.finite_rings.finite_field_base.FiniteField', 'sage.rings.number_field.number_field_base.NumberField', 'sage.rings.finite_rings.finite_field_constructor.FiniteFieldFactory', 'sage.rings.fraction_field.FractionField_generic']}
```

Similarly, we can do cycle detection. Notice that we need to set a time limit as finding cycles is, in general, NP-hard.
```
% sdeps -cc sage.rings.integer.Integer 120
[['sage.rings.integer.Integer', 'sage.rings.rational.Rational', 'sage.rings.integer_ring.IntegerRing_class', 'sage.rings.ring.CommutativeRing', 'sage.rings.polynomial.polynomial_element.Polynomial', 'sage.rings.rational_field.RationalField', 'sage.rings.number_field.number_field_base.NumberField', 'sage.rings.qqbar.AlgebraicRealField', 'sage.rings.qqbar.AlgebraicField_common', 'sage.rings.qqbar.AlgebraicNumberPowQQAction', 'sage.rings.real_mpfi.RealIntervalFieldElement', 'sage.rings.integer.Integer'], 
...
]
```

## As a package

There is more freedom of use when treated as a package. An example workflow is shown below.

```
# Import relevant files
from sagedeps.main import *
from sagedeps.deps.loader import Loader
from sagedeps.deps.data import *
from sagedeps.deps.model.dependency import *
from sagedeps.deps.filter import *
from sagedeps.analysis import *

# Load from resources
Loader.initialize(scorer=DefaultScorer())

# Get classes and directories
filter = EmptyFilter()
classes = Data.get_classes_filtered(
    filter
)
dirs = Data.get_modules_filtered(
    filter
)
print(len(dirs))
print(len(classes))

# Count number of outdegrees over all classes to get an average
total_deg = 0
for sc in classes:
    total_deg += sc.out_degree(
        relations=[Relation.INHERITANCE, Relation.CLASS_ATTRIBUTE, Relation.DECLARED_SUB_IMPORT, Relation.DECLARED_TOP_IMPORT]
    )

print(total_deg/len(classes))

# Analyze for cliques of size 3
analyzer = CliquesAnalyzer(
    filter=EmptyFilter(),
    size=3,
    edge_types=[Relation.INHERITANCE, Relation.CLASS_ATTRIBUTE, Relation.DECLARED_SUB_IMPORT, Relation.DECLARED_TOP_IMPORT]
)

# Analyze PageRank
analyzer = PageRankAnalyzer(
    filter=EmptyFilter(),
    edge_types=[Relation.INHERITANCE, Relation.CLASS_ATTRIBUTE, Relation.DECLARED_SUB_IMPORT, Relation.DECLARED_TOP_IMPORT],
    weights=[1,1,1,1]
)
res = analyzer.run()
print(res[:15])
print(len(res))
```

The above snippet outputs

```
3202
5854
3.3659036556200888
[('sage.structure.sage_object.SageObject', np.float64(0.037002533039364974)), ('sage.rings.integer.Integer', np.float64(0.02290838092408753)), ('sage.rings.integer_ring.IntegerRing_class', np.float64(0.02273003398295342)), ('sage.structure.element.Element', np.float64(0.02090351208515615)), ('sage.structure.parent.Parent', np.float64(0.01437274473122256)), ('sage.categories.category.Category', np.float64(0.011457959949590773)), ('sage.categories.category_singleton.Category_singleton', np.float64(0.010601666149250333)), ('sage.typeset.character_art.CharacterArt', np.float64(0.010176442750443552)), ('sage.misc.lazy_attribute.lazy_attribute', np.float64(0.008751776770612866)), ('sage.categories.category_with_axiom.CategoryWithAxiom', np.float64(0.008289012215276066)), ('sage.structure.element.ModuleElement', np.float64(0.008201175230707807)), ('sage.rings.rational_field.RationalField', np.float64(0.008088172337409227)), ('sage.misc.lazy_attribute._lazy_attribute', np.float64(0.008085176090604891)), ('sage.categories.objects.Objects', np.float64(0.007503707995711554)), ('sage.structure.unique_representation.UniqueRepresentation', np.float64(0.007267035285008635))]
5854
```