Package: postprocess-sdist-make-rust-ext-opt
postprocess-sdist-make-rust-ext-optThis is a small tool to "post-process" Python source distributions ("sdists")
containingsetuptools-rust-based Rust extensions so that these extensions
are marked as "optional" (cf.optionalparameter in thesetuptools-rustAPI docs).In other words, it takes an sdist with asetup.pythat looks like thisfromsetuptoolsimportsetupsetup(...rust_extensions=[RustExtension(...),]...)and turns it into one that looks like this:fromsetuptoolsimportsetupsetup(...rust_extensions=[RustExtension(...,optional=True),]...)What? Why?What does it mean for an extension to be optional?An extension (Rust or otherwise) being optional means that if the build fails
when installing the package, the installation of the remainder of the package
will proceed anyway and be considered successful. The package can then deal
with the extension's absence at runtime, e.g. by providing pure-Python
fallbacks for its functionality.Why set it as optional in a postprocessing step and not from the start?Because you'll probably want to build binary packages (wheels) from the project
as well, but if your extension is marked as optional, any errors during their
build will be ignored. So you don't generally want to have it set as optional
when building wheels. It only really makes sense to have it set for the sdist,
nothing else.Isn't there a way to do this dynamically without post-processing?There is, and in fact that is theapproach recommended by
cibuildwheel:
You just make theoptionalsetting in yoursetup.pydependent on some
environment variable that tells you whether a wheel is being built or not.I thought about similar approaches before I wrote this utility, but figured
you'd want to keep the number of "dynamic" values insetup.pyto a minimum so
it will be easy to port it to declarative alternatives likesetup.cfgorpyproject.tomleventually.
But being able to specifyanykind of extension (even C ones) declaratively
isnowhere close to being implemented in setuptools,
and once it is, this tool in its current form won't help you anyway because
it's justsetup.pyfocused (although that could be changed then), so, to be
honest, it's not much of an argument.If I had known about these things or thought them through properly, I probably
wouldn't have written it 🤷Back to postprocessing: Why not do it the other way round?Another option would be to set the extension as optional from the start but
change it to non-optional before the wheel build. But the issue with that is
that if you're using tools likesetuptools-scmthat automatically determine
your package's version from its state as determined by a version control system
(VCS) like Git, changing anything about the code will cause the version to be
considered "dirty", which will be represented in the version string. A way to
work around this would be to manipulate the VCS history in this case, but that
is even more of a hack than the postprocessing.Why not change it prior to building the sdist?The same reason as above (dirty repo when building => modified
automatically-determined version).Installationpip install postprocess-sdist-make-rust-ext-optUsage$postprocess-sdist-make-rust-ext-opt--help
Usage:postprocess-sdist-make-rust-ext-opt[OPTIONS]SDIST_PATHS...Post-processaPythonsdisttomakeitsRustextensionsoptional

Arguments:SDIST_PATHS...path(s)ofsdist.tar.gzarchive(s)topost-process[required]Options:-O,--output-dirDIRECTORYdirectoryinwhichtoplacethepost-processedarchives(default:directorynamed'postprocess'ininputsdistdirectory)-C,--create-output-dirwhethertocreatetheoutputdirectory(anditsparents)ifitdoesnotexist-f,--forcewhethertooverwriteexistingfilesintheoutputdirectory--install-completion[bash|zsh|fish|powershell|pwsh]Installcompletionforthespecifiedshell.--show-completion[bash|zsh|fish|powershell|pwsh]Showcompletionforthespecifiedshell,tocopyitorcustomizetheinstallation.--helpShowthismessageandexit.Unless otherwise specified, a processed sdist will be written to a folder namedpostprocessedin the same directory as the input sdist. Its filename will be
the same as that of the input sdist.To modify sdists in-place, you can use the combination of options-fO dir dir/*.tar.gz, wherediris the directory cotaining the sdists.CaveatsTheRustExtensioncalls for which theoptionalargument should be set toTruemustbe placed directly inside the list that is assigned to therust_extensionsparameter of the top-levelsetup()call like so:fromsetuptoolsimportsetupsetup(...rust_extensions=[RustExtension(...),]...)Anything more indirect than that, e.g. assigning aRustExtensioninstance to
a variable and then placing that inside therust_extensionslist, will cause
the tool to exit with an error.This is because the transformation is implemented at the syntax tree level and
no static analysis is performed to trace arguments back to their origins.AcknowledgementsThe transformation is performed usingRedBaron's full syntax tree (FST)
representation of the sdist'ssetup.py.