Package: pygranso-cpu
PyGRANSOPyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiationPlease checkhttps://ncvx.org/PyGRANSOfor detailed instructions (introduction, installation, settings, examples...).PyGRANSO is AGPL v3.0 licensed, but it also contains a small portion of GPL code.  Please see the LICENSE_INFO folder for more information.Brief IntroductionOptimizing nonconvex (NCVX) problems, especially nonsmooth and constrained ones, is an essential part of machine learning. However, it can be hard to reliably solve such problems without optimization expertise. Existing general-purpose NCVX optimization packages are powerful but typically cannot handle nonsmoothness. GRANSO is among the first optimization solvers targeting general nonsmooth NCVX problems with nonsmooth constraints, but, as it is implemented in MATLAB and requires the user to provide analytical gradients, GRANSO is often not a convenient choice in machine learning (especially deep learning) applications. To greatly lower the technical barrier, we introduce a new software package called NCVX, whose initial release contains the solver PyGRANSO, a PyTorch-enabled port of GRANSO incorporating auto-differentiation, GPU acceleration, tensor input, and support for new QP solvers. NCVX is built on freely available and widely used open-source frameworks, and as a highlight, can solve general constrained deep learning problems, the first of its kind.InstallationNOTE: The installation process is tested on Ubuntu 20.04Installing PyGRANSO is simple. Here is a step-by-step instruction:InstallPython >= 3.9(GPU and CPU) Install from PyPI:pip install pygranso==1.0.0.dev1 -f https://download.pytorch.org/whl/cu111/torch_stable.html(CPU only) Install from PyPI:pip install pygranso-cpu==1.0.0.dev1 -f https://download.pytorch.org/whl/cpu/torch_stable.htmlOptional Steps:Clone the most recent PyGRANSO package to get test examples:git clone https://github.com/sun-umn/PyGRANSO.git
 cd PyGRANSO(GPU and CPU) Run test to make sure the dependency installation is correct:python test_cuda.py(CPU only) Run test to make sure the dependency installation is correct:python test_cpu.pyCheck theexample folderin the source code orexample sectionon the documentation website to get started.DependenciesPython-3.9.7

numpy-1.20.3

scipy-1.7.1

pytorch-1.9.0

osqp-0.6.2

Jupyter Notebook-6.4.5Change LogsPlease checkCHANGELOG.mdin the main folder.Notes on DocumentationPyGRANSO is a PyTorch-enabled port of GRANSO with auto-differentiation, and some of its documentation uses MATLAB conventions. In the PyGRANSO source code docstrings, please note that:structrefers topygransoStruct, which is a simple class that users need to use for specifying their problems and options.Vector and matrix refer to PyTorch tensor with(n,1)and(m,n)dimension, respectively.AcknowledgementsWe would like to thankFrank E. CurtisandMichael L. Overtonfor their involvement in creating the BFGS-SQP algorithm that is
implemented in the software packageGRANSO. This work was supported by UMII Seed Grant Program and NSF CMMI 2038403.CitationIf you publish work that uses or refers to PyGRANSO, please cite the following two papers,
which respectively introduced PyGRANSO and GRANSO:[1] Buyun Liang, Tim Mitchell, and Ju Sun,
NCVX: A User-Friendly and Scalable Package for Nonconvex
Optimization in Machine Learning, arXiv preprint arXiv:2111.13984 (2021).Available athttps://arxiv.org/abs/2111.13984[2] Frank E. Curtis, Tim Mitchell, and Michael L. Overton,
A BFGS-SQP method for nonsmooth, nonconvex, constrained
optimization and its evaluation using relative minimization
profiles, Optimization Methods and Software, 32(1):148-181, 2017.Available athttps://dx.doi.org/10.1080/10556788.2016.1208749BibTex:@article{liang2021ncvx,
    title={{NCVX}: {A} User-Friendly and Scalable Package for Nonconvex 
    Optimization in Machine Learning}, 
    author={Buyun Liang, Tim Mitchell, and Ju Sun},
    year={2021},
    eprint={2111.13984},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

@article{curtis2017bfgssqp,
    title={A {BFGS-SQP} method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles},
    author={Frank E. Curtis, Tim Mitchell, and Michael L. Overton},
    journal={Optimization Methods and Software},
    volume={32},
    number={1},
    pages={148--181},
    year={2017},
    publisher={Taylor \& Francis}
}ContactFor questions or bug reports, please either:raise issues in the PyGRANSO repository (https://github.com/sun-umn/PyGRANSO/) orsend an email to:Buyun Liang (liang664 an_at_symbol umn a_dot_symbol edu)Tim Mitchell (tim an_at_symbol timmitchell a_dot_symbol com)Ju Sun (jusun an_at_symbol umn a_dot_symbol edu)Thanks to other contributors and bug reporters:Chen Jiang: Tested perceptual attack example (ex6). Prepared environment file for Win10.