Package: mmf-setup
mmf-setupThis meta-project provides an easy way to install all of the python tools I typically
use. It also serves as a fairly minimal example of setting up a package thatpipcan
install, and specifying dependencies.In particular, I structure it for the following use-cases:Rapid installation and configuration of the tools I need. For example, I often useCoCalc. Whenever I create a new project, I need to perform some
initialization. With this project, it is simply a matter of usingpipxto install
this package, and then using some of the tools. Specifically:pipxinstallmmf-setup
mmf_setupcocalcmercurialhg-evolvehg-gitblackThis does the following, and then links various configuration files to the home
directory:pipxinstallmmf-setup
pipxinjectmmf-setupmercurialhg-evolvehg-gitblack
curl-sSLhttps://install.python-poetry.org|python3-Initial setup of a python distribution on a new computer. This is a little more
involved since one needs to first install python (I recommend usingMiniconda) and then updating the tools.A place to document various aspects of installing and setting up python and related
tools. Some of this is old, but kept here for reference.A generic way of settingsys.pathfor development work using the following (in order
of decreasing precedence) by callingmmf_setup.set_path().An entry in apyproject.tomlfile somewhere in a higher-level directory.An entry in asetup.cfgfile somewhere in a higher-level directory.The first parent directory with either of these files, asetup.pyfile, or.hgor.gitdirectories.Quickstart (TL;DR)To get the notebook initialization features without having to install the package,
just copynbinit.pyto your project. Importing this will try to executeimport mmf_setup;mmf_setup.nbinit()but failing this, will manually run a similar
code.Install this package from the source directory,PyPI, etc. with one of the following:Directly from PyPIpython3-mpipinstall--usermmf-setup[nbextensions]From Sourcepython3-mpipinstall--user
hg+https://alum.mit.edu/www/mforbes/hg/forbes-group/mmf-setup[nbextensions]From Local Source(Run this from the source directory after you unpack it.)python3-mpipinstall--user.[nbextensions]Note: these includes thenbextensionsextra. You and run without the--userflag
if you want to install them system-wide rather than intosite.USER_BASE.To get the notebook tools for Jupyter (IPython) notebooks, execute the following as
a code cell in your notebook and then trust the notebook withFile/Trust Notebook:importmmf_setup;mmf_setup.nbinit()This will set the theme which is implemented in the output cell so it is stored for
use online such as when rendered throughNBViewer. One can specify different
themes. (Presently onlytheme='default'andtheme='mmf'are supported.)Mercurial:If you want to install mercurial with thehg-gitandEvolveextensions, then you can do that with thehgextra:python3-mpipinstall--user.[hg]This essentially runspip install mercurial hg-git hg-evolve.  You can then enable
these in your environment by sourcing themmf_setupscript in one of the following
ways:eval$"(mmf_setup -v)"# Enable hg lg but not evolve, etc.eval$"(mmf_setup -v -H)"# Enable evolve, etc.mmf_setup-v[-H]# Shows what will be set set (dry run).To do this automatically when you login, add this line to your~/.bashcor~/.bash_profilescripts.Warning:theeval $"(mmf_setup -v -H)"option also includes a mercurial
update hook which will add%include ../.hgrcto your projects.hg/hgrcfile
uponhg > update.  This allows you to include project-specific mercurial
customizations in your repository, but is a potential security risk.  See the
discussion below.These can also be enabled manually by adding the following to your~/.hgrcfile:# ~/.hgrc...[extensions]evolve=topics=hggit=Settingsys.pathThe preferred way to work with python code is to install it, preferably into a virtual
environment or a conda environment. By installing it, you will assure that your program
or library is available to notebooks, etc.  by being properly inserted into
sys.path. When developing, code, one can also do a"editable"
installswithpython -m install -e \<path\>so that code updates are seen.When developing code, however, this may not meet all use-cases, so we providemmf_utils.set_path(cwd='.')which will setmmf_setup.ROOTand insert it at the start
ofsys.path.The algorithm for determiningROOTis as follows starting fromcwd(which is'.'by default, but can be specified in the call toset_path()).  Note: in each case, the
specified path must be an existing directory or it will be ignored.A value specified asmmf_setup.ROOTwill override everything.A value in the environmental variableMMF_SETUP_ROOTwill override everything else.An explicitROOTentry in the firstpyproject.tomlfound:# pyproject.toml[tool.mmf_setup]ROOT='src'An explict entry in the firstsetup.cfg:# setup.cfg[mmf_setup]ROOT=srcThe first parent directory to contain any of the filessetup.py,setup.cfg, orpyproject.toml, or directories.git, or.hg.If none of these yields a valid existing directory forROOT, then it will not be set.Mercurial (hg) ToolsIf you source the output of themmf_setupscript with one of the following:eval$"(mmf_setup -v)"eval$"(mmf_setup -v -H)"then yourHGRCPATHwill be amended to includehgrc.lgaorhgrc.fullrespectively.  The first adds a usefulhg lga(hg lgfor short) command which provides a concise graphical display:$hglg
@200:dmichael(64minutesago)0.4.0[0.4.0]tip|ENH,DOC,TST,API:Cleanedupset_path...
o199:dmichael(31hoursago)0.4.0[0.4.0]|WIP,CHK:Cleaningupset_path.
o198:dmichael(20hoursago)0.4.0[0.4.0]|DOC:AddeddeveloperNotes.md
...The second version with-Hadds some useful extensions:hg-git,Evolve, and
enablesTopics.  The latter are required, for example, to interface withHeptapod.Finally, the-Hoption enables anhg updatehook, which adds%include ../.hgrcto
your project's.hg/hgrcfile when you callhg update.  This is a potential security
risk, because an untrusted repo could include dangerous commands in.hgrc.  Thus, we
require user intervention before including this:$eval$"(mmf_setup -v -H)"$hgclonehttps://alum.mit.edu/www/mforbes/hg/forbes-group/mmf-setup
...
updatingtobranchdefault88filesupdated,0filesmerged,0filesremoved,0filesunresolved
$cdmmf-setup
$hgup0.4.0
Repository.../mmf-setupcontainsan.hgrcfile.
Includingthisin.hg/hgrcisapotentialsecurityrisk.
Onlydothisifyoutrusttherepository.

Includethis.hgrcin.hg/hgrc?[yN]y
Adding'%include ../.hgrc'to'.hg/hgrc'...
$tail.hg/hgrc
...
%include../.hgrcCoCalcWe provide some tools for working onCoCalc.  To get started, simply do to the
following once you have enabled network access on your project by applying a license:pipxinstallmmf-setup
mmf_setupcocalcTo see exactly what this will do, run:mmf_setupcocalc-vOnce this is done, you should ensure that, whenever you use a terminal onCoCalc,
you appropriately set the following variables:LC_HG_USERNAME: Used bymercurial.LC_GIT_USEREMAIL,LC_GIT_USERNAME: Used bygit.These will identify you as you when you commit your work.  (Needed, because all users
share the same project.  Seecocalc#370for details.)I recommend the following:Forward these variables in your SSH config file (~/.ssh/config):# ~/.ssh/config
Host cc-project1
  User ff1cb986f...

Host cc*
  HostName ssh.cocalc.com
  ForwardAgent yes
  SetEnv LC_HG_USERNAME=Your Full Name <your.email.address+hg@gmail.com>
  SetEnv LC_GIT_USERNAME=Your Full Name
  SetEnv LC_GIT_USEREMAIL=your.email.address+git@gmail.com
  SetEnv LC_EDITOR=viThe appropriate value forUsercan be found in the project Settings onCoCalc.Optionally, useSendEnv ...and set these on your personal computer:# ~/.bashrc or similarLC_HG_USERNAME=YourFullName<your.email.address+hg@gmail.com>LC_GIT_USEREMAIL=your.email.address+git@gmail.comLC_GIT_USERNAME=YourFullNameIf you want to runGitorMercurialfrom theCoCalcweb interface, then
create a named terminal -- i.e.Michael.term-- and then set these variables in the
terminal startup script.  (Seehttps://doc.cocalc.com/terminal.html#startup-filesfor
details.)NotesSeeNotes.mdfor developer notes.  Other notes about python, IPython,
etc. are stored in thedocsfolder.Changes0.4.11Fix bug with warning on CoCalc.0.4.10Add warning on CoCalc if users have not setLC_HG_USERNAME, orLC_GIT_USERNAME.0.4.9Remove blank lines fromdefault.texso thatmmf_setup.nbinit()works on CoCalc
(workaround forCoCalc issue #6210).0.4.8Don't install mercurial etc. on CoCalc by default: these are now included in the
Default Ubuntu 20.04 image.  If you need these, then explicitly specify them:mmf-setupcocalcmercurialhg-githg-evolvejupytextblack0.4.7Recommend and support installing mmf-setup with pipx on CoCalc.Use new Poetry install.Drop support for python3.6.0.4.6Use python3 for CoCalc setup.  (Is a problem on Docker images).Ignore matplotlib errors.0.4.3Resolves issue #28: Failure ofmmf_initial_setupon Mac OS X if .DS_Store files exist.Resolves issue #29: Relative paths in config files resolve to absolute pathsAdded aliasesfindpy,findf,findaetc. for CoCalc for runningfind+grep.0.4.2Resolves issue #26: Significant improvements for installation on CoCalc.Moves~/.bashrcto~/.bashrc_cocalcso we can insert~/.local/binto thePATHfor non-login shells.  This is important when trying to push withhgover
SSH, which runshgwithout logging in.  Our new workflow needs thathgto have
access toevolveetc.   Previously this worked by updating these with the systemPython 2version ofhg, but that now fails with new projects, so we have removed
this.Add.hgrctweaks to top-level cocalc~.hgrc(might as well includehgrc.lgahere now so students etc. can see these.  Don't enableevolveetc. by default
though.)Users should now calleval "$(mmf_setup -v -H)"rather than using source. mmf_setup -H.  Sourcingmmf_setupnow emits a deprecation warning message.Cocalc install is now tested in an isolated manner.  (Only thenox -s test_cocalcworks... see issue #27.)Makehgrc.fullPOSIX compliant (work withsh).  (Was causing GitHub workflow to
fail.)  Scripts still requirebashwhich is explicitly chosen when we callrun-tests.py.Working GitHub workflows and cocalc tests.Updated release process.0.4.1Makemmf_setup -voutput clean so that we caneval "$(mmf_setup -v)"rather than
sourcing.In the future we will make this a python file, soevalwill be the only
option.Fixed issues withnbinit()failing with not paths defined.Added more tests.0.4.0Resolves issue #23: Drop support of nbclean mercurial extension (to hard to
maintain), thereby making issues #1, #3, #8, #9, #11, obsolete.Remove installation of old obsolete notebook extensions like Calico tools,RISEetc. These can all be
installed now with either the standard extensions, or pip.With pip:python-mpipinstalljupyter_contrib_nbextensionsRISE
jupytercontribnbextensioninstall--userWith conda:condainstall-cconda-forgejupyter_contrib_nbextensionsrise
jupytercontribnbextensioninstall--userSee alsoNBExtensions.Improved test isolation.Expanded options formmf_setup.Mercurial configuration is now in two stages:hgrc.lgawhich gives thehg lgcommand safely using only builtin modules, andhgrc.fullwhich includes this,
and also activates thehg-gitandevolveextensions.  This also applies theupdatehook for including.hgrcfiles, but asks the user for confirmation
because this is a security risk.0.3.1Addedmmf_setup cocalcinitialization for CoCalc.com projects
(changed frommmf_setup smc).Add message tonbinit()call indicating that HGROOT has been added
tosys.path.Resolves issue #22 with git username classes on CoCalc.0.3.0Support python 3 - drop support for python 2.Conda installable from mforbes channel on anaconda cloud.Add missing default.tex theme so that nbinit() brings in math definitions even if no
theme is specified.Fixed KeyError:asyncioerror on failed IPython import.0.1.13Incomplete version... everything here is rolled into 0.3.00.1.12Mademmf_initial_setuppython 3 compliant.Added logging tonbinit()and made some slight tweaks toHGROOT.Added\D,\sn,\cn,\dncommands for MathJaX.0.1.11Resolve issue #20:mmf_setup.set_path.set_path_from_fileallows for configuration of path insetup.cfgfile.Fix python 3 bug: TypeError: Can't mix strings and bytes in path components0.1.10Added better backwards compatibility for previous changes.Simplified nbinit theme management and use'default'theme as default.New themes only need to overwrite what they need.Don't change fonts as default since this does not work well on CoCalc (the code
cells change size on clicking which is a pain.)0.1.9Resolve issues:#17: storemmf_setup.HGROOT#18: safer exception handling withnbcleancommands so data is not lost#19:nbcleanworks with new mercurial APIAdded\erfi,\sech,\Braketimport mmf_setup.set_path.hgrootwill addHGROOTto path without IPythonAdded standalonenbinit.pyfor use withoutmmf_setupinstalls.0.1.8Resolves issue #15.Use$BASH_SOURCE{0}to getBIN_DIR(resolves issue #16)Addednbinit(hgroot)option to addhg roottosys.path(issue #13)0.1.7Changed option to--clean-all(resolves issue #7)Updated the notebook style (added\Tr, fixed output overflow issue)Added pip option mmf_setup[nbextensions]Removed'EnableNBExtensionApp'dependence which broke new Jupyter (resolves issue
#12)Added some files for initializing setup on Sage Mathcloud (SMC) (resolves issue #14).Addedmmf_initial_setupscript and some init files (.inputrc,.hgrc,.bash_aliases).Run withmmf_setup smcRemoved old extension manager since nbextensions are quite good now.0.1.6Added cupdate command (resolves issue #4)Fixed bug with ccommit where it did not accept a list of files
(issue #6)Issue commands in a context to provide a robust mechanism for
recovery (issue #5)