Package: pishield
PiShield: A NeSy Framework for Learning with Requirements:sparkles:Description:pushpin:Dependencies:hammer_and_wrench:Installation:bulb:UsageInference timeTraining time:arrow_forward:Demo video:fire:Performance1. Autonomous Driving2. Tabular Data Generation3. Functional GenomicsTeam:memo:References:sparkles: DescriptionUpdate: PiShield'swebsiteis now availablehere.PiShield is the first framework ever allowing for the integration of the requirements into the neural networks' topology.:white_check_mark: The integration happens in a straightforward and efficient manner and allows for the creation of deep learning models that are guaranteed to be compliant with the given requirements, no matter the input.:pencil2: The requirements can be integrated both at inference and/or training time, depending on the practitioners' needs.:pushpin: DependenciesPiShield requires Python 3.8 or later and PyTorch.Optional step: conda environment setup, using cpu-only PyTorch here. Different PyTorch versions can be specified following the instructionshere.conda create -n "pishield" python=3.11 ipython 
conda activate pishield

conda install pytorch cpuonly -c pytorch 
pip install -r requirements.txt:hammer_and_wrench: InstallationFrom the root of this repository, containingsetup.py, run:pip install .:bulb: UsageAssume we have the following constraints and ordering of the variables in a fileexample_constraints_tabular.txt:ordering y_0 y_1 y_2
-y_0 >= -3
y_0 >= 3
y_0 - y_1 >= 0
-y_0 - y_2 >= 0Inference timeTo correct predictions at inference time such that they satisfy the constraints, we can use PiShield as follows:import torch
from pishield.constraint_layer import build_pishield_layer

predictions = torch.tensor([[-5., -2., -1.]])
constraints_path = 'example_constraints_tabular.txt'

num_variables = predictions.shape[-1]
CL = build_pishield_layer(num_variables, constraints_path)

# apply CL from PiShield to get the corrected predictions
corrected_predictions = CL(predictions.clone())  # returns tensor([[ 3., -2., -3.]], which satisfies the constraintsimport torch
from pishield.constraint_layer import build_pishield_layer

def correct_predictions(predictions: torch.Tensor, constraints_path: str):
    num_variables = predictions.shape[-1]
    
    # build a constraint layer CL using PiShield
    CL = build_pishield_layer(num_variables, constraints_path)
    
    # apply PiShield to get corrected predictions, which satisfy the constraints
    corrected_predictions = CL(predictions)
    return corrected_predictionsTraining timeAssume a Deep Generative Model (DGM) is used to obtain synthetic tabular data.
Using PiShield at training time is easy, as it requires two steps:Instantiating the ConstraintLayer class from PiShield in the DGM's constructor.Applying the ConstraintLayer on the generated data obtained from the DGM before updating the loss function of the DGM.:arrow_forward: Demo videoA demo video is available for downloadhere.:fire: Performance1. Autonomous DrivingIn [2], we considered standard 3D-RetinaNet models with different temporal learning architectures such as I3D, C2D, RCN, RCGRU, RCLSTM, and SlowFast, and compared each of these with their constrained versions.
The constrained versions inject propositional background knowledge into the models via a constrained layer, as we call it in [2], which is equivalent to a Shield layer when using PiShield.Below we report the aggregated performance from Table 2 of our paper [2] to show the results we obtained according to the f-mAP (framewise mean Average Precision) measure, at IOU (Intersection-over-Union) threshold of 0.5.
The best results are inbold.As we can see, the models incorporating background knowledge throughShield layersoutperform their standard counterparts.BaselineShieldedI3D29.3030.98C2D26.3427.93RCN29.2630.02RCGRU29.2430.50RCLSTM28.9330.42SlowFast29.7331.88___________AVERAGE28.8030.292. Tabular Data GenerationIn [1], we compared standard deep generative models with their respective constrained versions, which use linear inequality constraints.
The latter are the models to which we added a constraint layer, as we call it in [1], which is equivalent to a Shield layer when using PiShield.Below we reproduce Table 2 of our paper [1] to show the results we obtained according to two standard measures for tabular data generation benchmarks: utility and detection.
For each of these two measures, we report the performance using three metrics: F1-score (F1), weighted F1-score (wF1), and Area Under the ROC Curve (AUC).
The best results are in bold.As we can see, in 28/30 cases, the models incorporating background knowledge throughShield layersoutperform their standard counterparts.Utility(↑)Detection(↓)F1wF1AUCF1wF1AUCWGAN0.4630.4880.7300.9450.9430.954Shielded-WGAN0.4830.5020.7450.9150.9120.934TableGAN0.3300.4000.7040.9080.9070.926Shielded-TableGAN0.3750.4320.7140.8980.8950.917CTGAN0.5170.5320.7710.9020.9010.920Shielded-CTGAN0.5160.5370.7730.8940.8910.919TVAE0.4970.5270.7670.8690.8680.892Shielded-TVAE0.5070.5370.7730.8680.8670.898GOGGLE0.3440.3730.6240.9260.9260.943Shielded-GOGGLE0.4090.4270.6670.9250.9160.937________________AVERAGE Baseline0.4300.4640.7190.9100.9090.927AVERAGE Shielded0.4580.4870.7340.9000.8960.9213. Functional GenomicsIn our paper [3], we compared the performance of baseline models with their constrained counterparts.
The latter are the models to which we added a constraint layer, as we call it in [3], which is equivalent to a Shield layer when using PiShield.Below we aggregated the results from Table 3 of [3] and reported the performance in terms of the area under the average precision and recall curve (AU(PRC)), which is a standard metric used in functional genomics.
As we can see, the modelsShield layersoutperform their standard counterparts.DatasetBaseline*PiShieldCELLCYCLE0.2200.232DERISI0.1790.182EISEN0.2620.285EXPRE0.2460.270GASCH10.2390.261GASCH20.2210.235SEQ0.2450.274SPO0.1860.190__________AVERAGE0.2250.241*Note: All baselines for the functional genomics scenario have a postprocessing step included, as functional genomics tasks always require that the constraints are satisfied.TeamMihaela Catalina Stoian[GitHub][LinkedIn][Google Scholar]Alex Tatomir[GitHub][LinkedIn][Google Scholar]Thomas Lukasiewicz[Google Scholar]Eleonora Giunchiglia[GitHub][LinkedIn][Google Scholar]:memo: References[1] Mihaela Catalina Stoian, Salijona Dyrmishi, Maxime Cordy, Thomas Lukasiewicz, Eleonora Giunchiglia. How Realistic Is Your Synthetic Data? Constraining Deep Generative Models for Tabular Data. arXiv:2402.04823. Accepted at the International
Conference on Learning Representations (ICLR), 2024.[2] Eleonora Giunchiglia, Alex Tatomir, Mihaela Catalina Stoian, Thomas Lukasiewicz. CCN+: A neuro-symbolic framework for deep learning with requirements. International Journal of Approximate Reasoning, 2024.[3] Eleonora Giunchiglia and Thomas Lukasiewicz. Coherent Hierarchical Multi-Label Classification Networks. In Proceedings of Neural
Information Processing Systems, 2020.