Package: epoch8-tqdm-loggable
tqdm-loggabletqdm-loggableis a petite Python package providing logging friendly TQDM progress bars.If your Python application hastqdmprogress bars and you use them in a non-interactive session like...Background workerDocker containerEdge computingLogstash,Sentry,Datadogor other external log toolsLong-running machine learning tasks...orstdoutstream is otherwise not available or redirected...you cannot have interactive progress bar. What happens is that if you are observing
your application using monitoring tools, you usually do not see anything happening while your
application is havingtqdmprogress ongoing. If the progress bar'ed operation takes few minutes your appliaction
may appear frozen. This is  fixed bytqdm-loggingby sending a regular reports about your progress to logging backend like files and log monitoring
tools.In these situationstqdm-loggablewill automatically turn yourtqdmprogress bars to loggable progress messages
that can be read in headless systems.tqdm-loggable...Is a drop-in replacement for the normaltqdm- nothing changes unless non-interactive session is detectedUses Pythonloggingsubsystem to report status instead of terminalPrint a log line for every X secondsThe logging messages are structured, so they work with Sentry, LogStash, etc. rich logging services
which provide advanced searching and tagging by variablesSpecial support for Github Actions and other continous integration environmentsHere is a sampletqdmlog message output in plain text logs:tqdm_logging.py     :139  2022-09-21 12:13:44,138 Progress on:Progress bar without total -/- rate:- remaining:? elapsed:00:00 postfix:-
tqdm_logging.py     :139  2022-09-21 12:13:46,225 Progress on:Progress bar without total 10000/- rate:- remaining:? elapsed:00:02 postfix:-

tqdm_logging.py     :139  2022-09-21 12:13:46,225 Progress on:Sample progress -/60000 rate:- remaining:? elapsed:00:00 postfix:-
tqdm_logging.py     :139  2022-09-21 12:13:56,307 Progress on:Sample progress 21.0kit/60.0kit rate:1,982.9it/s remaining:00:19 elapsed:00:10 postfix:Current time=2022-09-21 10:13:55.801787
tqdm_logging.py     :139  2022-09-21 12:14:06,392 Progress on:Sample progress 41.0kit/60.0kit rate:1,984.1it/s remaining:00:09 elapsed:00:20 postfix:Current time=2022-09-21 10:14:05.890220Note thattqdm-loggableis not to be confused withtqdm.contrib.loggingthat is very different approach for a different problem.InstallationThe package name istqdm-loggable.Read Python packaging manualon how to install packages
on your system.UsageThe only things you need to doMake sure yourPython logging system is properly configuredChange import fromfrom tqdm.auto import tqdmtofrom tqdm_loggable.auto import tqdmOptionally calltqdm_logging.set_level()at the init of your applicationOptionally calltqdm_logging.set_log_rate()at the init of your applicationSearch and replace instructions for your Python codebase:from tqdm import tqdm -> from tqdm_loggable.auto import tqdm 
from tqdm.auto import tqdm -> from tqdm_loggable.auto import tqdmHere isan example script:importdatetimeimportloggingimporttimefromtqdm_loggable.autoimporttqdmfromtqdm_loggable.tqdm_loggingimporttqdm_logginglogger=logging.getLogger(__name__)defmain():fmt=f"%(filename)-20s:%(lineno)-4d %(asctime)s %(message)s"logging.basicConfig(level=logging.INFO,format=fmt,handlers=[logging.StreamHandler()])# Set the log level to all tqdm-logging progress bars.# Defaults to info - no need to set if you do not want to change the leveltqdm_logging.set_level(logging.INFO)# Set the rate how often we update logs# Defaults to 10 seconds - optionaltqdm_logging.set_log_rate(datetime.timedelta(seconds=10))logger.info("This is an INFO test message using Python logging")withtqdm(total=60_000,desc="Sample progress",unit_scale=True)asprogress_bar:foriinrange(60_000):progress_bar.update(1000)# Test postfix outputprogress_bar.set_postfix({"Current time":datetime.datetime.utcnow()})time.sleep(0.5)tqdm_loggablewill detect non-interactive sessions.
If the application is running without a proper terminal, non-interactive progress messages will be used.
Otherwise progress bar is delegatedtqdm.automodule to maintain the compatibility
with anytqdmsystem without any changes to code.tqdm_loggablehas also progress bar workarounds forJupyter Notebook environmentslikeDatalorewhichare not 100% compatiblewith the original Jupyter Notebook.The Python logger instance used to log the messages is namedtqqm_loggable.tqm_logging.DevelopmentYou can usetqdm_loggable/manual_tests.pyto run the various checks
to see what different interactive and non-interactive sessions give for you.# Normal interactive terminal runpoetryrunmanual-testsBecause this is a normal shell session you will get a normal progress bar:Sample progress:  20%|████████▏                                | 12.0k/60.0k [00:05<00:24, 1.98kit/s, Current time=2022-09-21 15:40:24.274670]...then test without without a properTERM environment variable:# Disable interactive terminal by fiddling with TERM environment variableTERM=dumbpoetryrunmanual-testsYou get log messages:tqdm_logging.py     :139  2022-09-21 17:41:20,720 Progress on:Sample progress -/60000 rate:- remaining:? elapsed:00:00 postfix:-
tqdm_logging.py     :139  2022-09-21 17:41:30,803 Progress on:Sample progress 21.0kit/60.0kit rate:1,984.7it/s remaining:00:19 elapsed:00:10 postfix:Current time=2022-09-21 15:41:30.300714...or with different Docker sessions:# This will display process as log messagesdockerbuild-tmanual-tests.&&dockerrunmanual-tests# This will allocate a terminal and display progress as a normal tqdm progress bardockerbuild-tmanual-tests.&&dockerrun-timanual-testsor with redirected stdout:poetryrunmanual-tests>output.txt
catoutput.txtThese will output our terminal detection info and draw a progress bar, total 30 seconds.tqdm-loggable manual tests
sys.stdout.isatty(): False
TERM: -
is_interactive_session(): Falseand further progress bar or progress messages will follow depending
if you run the manual test interactively or not.See alsoSee other relevant logging packages:tqdm-loggable on PyPipython-discord-logging-handlerpython-logstashKudosOriginally build forTrading Strategy blockchain trade automation.See the original StackOverflow question.LicenseMIT