Package: apt-smart
简体中文Theapt-smartpackage automates robustapt-getmirror (a.k.a Repositories, Sources) selection forDebian,UbuntuandLinux Mintby enabling smart discovery of available mirrors, smart ranking of
available mirrors, automatic switching between mirrors and robust package list
updating (seefeatures). It’s currently tested on Python 2.7, 3.4, 3.5,
3.6, 3.7, 3.8 and PyPy (although test coverage is still rather low, seestatus).Why?FeaturesStatusInstallationUsageIssues with mirror updatesContactLicenseWhy?As a successor ofapt-mirror-updater,apt-smarthas many improvements in intelligence, speed, accuracy and robustness (seechangelog) when offering the best mirror for you.
It has a plan to optionally be a set-and-forget smart daemon: running in the background as a reverse proxy
always redirecting to the best mirror without root privilege. It also has a plan to support other distros like: Linux Mint (Done!) , ROS…FeaturesSmart discovery of available mirrorsDebian,UbuntuandLinux Mintmirrors are discovered automatically by querying theDebian mirror listor theUbuntu
mirror list1or theUbuntu
mirror list2or theLinux Mint mirror list(the applicable
mirror list is automatically selected based on the current platform).
It can smartly get mirrors within the country which the user is in.Smart ranking of available mirrorsDiscovered mirrors are ranked by bandwidth (to pick the fastest mirror) and whether they’re up-to-date and
excluded if they’re being updated (seeissues with mirror updates). e.g. with–list-mirrorsflag it would output like this:-----------------------------------------------------------------------------------------------------|Rank|MirrorURL|Available?|Updating?|Lastupdated|Bandwidth|-----------------------------------------------------------------------------------------------------|1|http://archive.ubuntu.com/ubuntu|Yes|No|Uptodate|16.95KB/s||2|http://mirrors.cqu.edu.cn/ubuntu|Yes|No|3hoursbehind|427.43KB/s||3|http://mirrors.nju.edu.cn/ubuntu|Yes|No|5hoursbehind|643.27KB/s||4|http://mirrors.tuna.tsinghua.e...|Yes|No|5hoursbehind|440.09KB/s||5|http://mirrors.cn99.com/ubuntu|Yes|No|13hoursbehind|2.64MB/s||6|http://mirrors.huaweicloud.com...|Yes|No|13hoursbehind|532.01KB/s||7|http://mirrors.dgut.edu.cn/ubuntu|Yes|No|13hoursbehind|328.25KB/s||8|http://mirrors.aliyun.com/ubuntu|Yes|No|23hoursbehind|1.06MB/s||9|http://ftp.sjtu.edu.cn/ubuntu|Yes|No|23hoursbehind|647.2KB/s||10|http://mirrors.yun-idc.com/ubuntu|Yes|No|23hoursbehind|526.6KB/s||11|http://mirror.lzu.edu.cn/ubuntu|Yes|No|23hoursbehind|210.99KB/s||12|http://mirrors.ustc.edu.cn/ubuntu|Yes|Yes|8hoursbehind|455.02KB/s||13|http://mirrors.sohu.com/ubuntu|No|No|Unknown|90.28bytes/s|-----------------------------------------------------------------------------------------------------Automatic switching between mirrorsThe main mirror configured in/etc/apt/sources.listcan be changed with a
single command. The new (to be configured) mirror can be selected
automatically or configured explicitly by the user.Robust package list updatingSeveralapt-getsubcommands can fail if the current mirror is being updated
(seeissues with mirror updates) andapt-smarttries to work
around this by wrappingapt-getupdateto retry on failures and
automatically switch to a different mirror when it looks like the current
mirror is being updated (because I’ve seen such updates take more than 15
minutes and it’s not always acceptable to wait for so long, especially in
automated solutions).StatusOn the one hand theapt-smartpackage was developed based on quite a
few years of experience in usingapt-getonDebianandUbuntusystems. On the
other hand the Python package itself is relatively new: it was developed and
published in Sep 2019. As such:WarningUntilapt-smarthas been rigorously tested I consider
it a proof of concept (beta software) so if it corrupts your
system you can’t complain that you weren’t warned! The worst that can happen
(assuming you trust my judgement ;-) is that/etc/apt/sources.listis corrupted however a backup copy is
made before any changes are applied, so I don’t see how this can
result in irreversible corruption.I’m working on an automated test suite but at the moment I’m still a bit fuzzy
on how to create representative tests for the error handling code paths (also,
writing a decent test suite requires a significant chunk of time :-).InstallationTheapt-smartpackage is available onPyPIwhich means installation
should be as simple as (paste all below commands together into terminal):sudoaptupdatesudoaptinstallpython-pippython-setuptoolspython-wheel-y# install python-pip and so on without askingpipinstall--userapt-smart# --user flag means install to per user site-packages directory(see below)echo"export PATH=\$(python -c 'import site; print(site.USER_BASE + \"/bin\")'):\$PATH">>~/.bashrcsource~/.bashrc# set per user site-packages directory to PATHThere’s actually a multitude of ways to install Python packages (e.g. theper
user site-packages directory,virtual environmentsor just installing
system wide) and I have no intention of getting into that discussion here, so
if this intimidates you then read up on your options before returning to these
instructions ;-).If a new version of apt-smart has been released, you can upgrade it via:pipinstall--userapt-smart--upgradeNote.apt-smartis ahelperfor theapttool. It isNOTareplacementforapt(or forapt-get). So,apt-smartshouldnotbe runinsteadof either of those commands. Nor shouldapt-smartbe run withsudoor viasu; ifapt-smarthappens to need root privilege in order for it to continue (in order
that it may, for example, changesources.list), then it will prompt
for a password.UsageThere are two ways to use theapt-smartpackage: As the command line
programapt-smartand as a Python API. For details about the
Python API please refer to the API documentation available onRead the Docs.
The command line interface is described below.Usage:apt-smart [OPTIONS]The apt-smart program automates robust apt-get mirror selection for
Debian and Ubuntu by enabling discovery of available mirrors, ranking of
available mirrors, automatic switching between mirrors and robust package list
updating.Supported options:OptionDescription-r,--remote-host=SSH_ALIASOperate on a remote system instead of the local system. TheSSH_ALIASargument gives the SSH alias of the remote host. It is assumed that the
remote account has root privileges or password-less sudo access.-f,--find-current-mirrorDetermine the main mirror that is currently configured in
/etc/apt/sources.list and report its URL on standard output.-F,--file-to-read=local_file_absolute_pathRead a local absolute path (path and filename must NOT contain whitespace) file
containing custom mirror URLs (one URL per line) to add custom mirrors to rank.-b,--find-best-mirrorDiscover available mirrors, rank them, select the best one and report its
URL on standard output.-l,--list-mirrorsList available (ranked) mirrors on the terminal in a human readable format.-L,--url-char-len=intAn integer to specify the length of chars in mirrors’ URL to display when
using--list-mirrors, default is 34-c,--change-mirror=MIRROR_URLUpdate /etc/apt/sources.list to use the givenMIRROR_URL.-a,--auto-change-mirrorDiscover available mirrors, rank the mirrors by connection speed and update
status and update /etc/apt/sources.list to use the best available mirror.-u,--update,--update-package-listsUpdate the package lists using “apt-get update”, retrying on failure and
automatically switch to a different mirror when it looks like the current
mirror is being updated.-U,--ubuntuUbuntu mode for Linux Mint to deal with upstream Ubuntu mirror instead of Linux Mint mirror.
e.g.--auto-change-mirror--ubuntuwill auto-change Linux Mint’s upstream Ubuntu mirror-x,--exclude=PATTERNAdd a pattern to the mirror selection blacklist.PATTERNis expected to be
a shell pattern (containing wild cards like “?” and “*”) that is matched
against the full URL of each mirror.-v,--verboseIncrease logging verbosity (can be repeated).-V,--versionShow version number and Python version.-R,--create-chroot=local_dir_absolute_pathCreate chroot with the best mirror in a local directory with absolute_path-q,--quietDecrease logging verbosity (can be repeated).-h,--helpShow this message and exit.Note: since apt-smart usesurlopenmethod in The Python Standard Library,you can set Environment Variables to make apt-smart connect via HTTP proxy, e.g. in terminal type:
export {http,https,ftp}_proxy=’http://user:password@myproxy.com:1080’
These will not persist however (no longer active after you close the terminal),
so you may wish to add the line to your ~/.bashrcIssues with mirror updatesThe most frequent failure that we run into isapt-getupdatecrapping out
with ‘hash sum mismatch’ errors (see alsoDebian bug #624122). When this
happens a file calledArchive-Update-in-Progress-*can sometimes be found
on the index page of the mirror that is being used (see alsoDebian bug
#110837). I’ve seen these situations last for more than 15 minutes.My working theory about these ‘hash sum mismatch’ errors is that they are
caused by the fact that mirror updates aren’t atomic, apparently causingapt-getupdateto download a package list whose datafiles aren’t consistent
with each other. If this assumption proves to be correct (and also assuming
that different mirrors are updated at different times :-) then the commandapt-smart--update-package-listsshould work around this annoying
failure mode (by automatically switching to a different mirror when ‘hash sum
mismatch’ errors are encountered).Publishingapt-smartto the world is my attempt to contribute to
this situation instead of complaining in bug trackers (see above) where no
robust and automated solution is emerging (at the time of writing). Who knows,
maybe some day these issues will be resolved by moving logic similar to what
I’ve implemented here intoapt-getitself. Of course it would also help if
mirror updates were atomic…ContactThe latest version ofapt-smartis available onPyPIandGitHub.
The documentation is hosted onRead the Docsand includes achangelog. For
bug reports please create an issue onGitHub.LicenseThis software is licensed under theMIT license.© 2020 martin68© 2018 Peter Odding.