Package: lognflow
lognflowLog and Flow tracking made easy with Python. You can install it by::pip install lognflowA simple program to use it would be similar to the following::from lognflow import lognflow
import numpy as np

data = np.random.rand(100)

logger = lognflow(r'c:\all_logs\\')
logger('This is a test for lognflow and log_single')
logger.log_single('data', data)The logviewer is also very useful::from lognflow import logviewer
logged = logviewer(r'c:\all_logs\some_log\\')
data = logged.get_single('data')The printprogress makes a pretty nice progress bar::from lognflow import printprogress
N = 100
pbar = printprogress(N)
for _ in range(N):
	# do_something()
	pbar()There is also a conviniant way to use multiprocessing in Python. You wish to
provide a function name, iterable inputs and shared inputs. Then ask
to run the function over the iterable inputs using multiprcessing. Then
The multiprocessor is for you. The following is a cross-correlation of two
masked verctors using a given ststistical function::from lognflow import multiprocessor

def masked_cross_correlation(inputs_to_iter_sliced, inputs_to_share):
	""" Calculate the correlation of two masked vectors, then use a given
		statsitcal function to get the stat of the correlation.
	"""
    vec1, vec2 = inputs_to_iter_sliced
    mask, statistics_func = inputs_to_share
    vec1 = vec1[mask==1]
    vec2 = vec2[mask==1]
    vec1 -= vec1.mean()
    vec1_std = vec1.std()
    if vec1_std > 0:
        vec1 /= vec1_std
    vec2 -= vec2.mean()
    vec2_std = vec2.std()
    if vec2_std > 0:
        vec2 /= vec2_std
    correlation = vec1 * vec2
    to_return = statistics_func(correlation)
    return(to_return)
    

if __name__ == '__main__':
    data_shape = (1000, 2000)
    data1 = np.random.randn(*data_shape)
    data2 = 2 + 5 * np.random.randn(*data_shape)
    mask = (2*np.random.rand(data_shape[1])).astype('int')
    statistics_func = np.median
    
    inputs_to_iter = (data1, data2)
    inputs_to_share = (mask, statistics_func)
    ccorr = multiprocessor(masked_cross_correlation, 
    					   inputs_to_iter, 
    					   inputs_to_share)
    print(f'ccorr: {ccorr}')In this package we use a folder on the HDD to generate files and folders in typical
formats such as numpy npy and npz, png, ... to log. A log viewer is also availble
to turn an already logged flow into variables. Obviously, it will read the folders
and map them for you, which is something you could spend hours to do by yourself.
Also there is the nicest progress bar, that you can easily understand
and use or implement yourself when you have the time.Looking at most logging packages online, you see that you need to spend a lot of time
learning how to use them and realizing how they work. Especially when you have to deal
with http servers and ... which will be a huge pain when working for companies
who have their own HPC.This is why lognflow is handy and straight forward.Many tests are avialable in the tests directory.Free software: GNU General Public License v3Documentation:https://lognflow.readthedocs.io.Featureslognflow puts all the logs into a directory on your pclognflow makes it easy to log text or simple plots.logviewer makes it easy to load variables or directoriesprintprogress is one of the best progress bars in Python.Credits
^^^^^^^^This package was created with Cookiecutter_ and theaudreyr/cookiecutter-pypackage_ project template... _Cookiecutter:https://github.com/audreyr/cookiecutter.. _audreyr/cookiecutter-pypackage:https://github.com/audreyr/cookiecutter-pypackageHistory0.1.0 (2022-11-16)First release on PyPI.0.3.0 (2022-12-19)Very consistent and easy-to-handle interface0.3.2 (2022-12-27)log your dictionary.0.5.2 (2023-01-11)logger call takes txt only for the main_logmain_log name can have no time_stampAll logs can be force_flush = True.delis well implemented.0.5.3 (2023-03-19)use logger.log_single('aName', aDict) to save a dictionary using numpy.saveztime_in_file_name has been changed to time_tag, sorry there! but early stages.time_tag is True in the constructor, but could always be set to FalseAll tests passed!0.6.0 (2023-03-22)This is a stable releaseAll text files are handled by the with statementRenaming bug is fixedall tests run properly.lognflow has all that logviewer has. We will check if dir exists at every use0.6.1 (2023-03-22)rename had a bug that is fixed0.6.2 (2023-03-25)made it possible to flush_all()We support Python 3.7+ because of dataclassesprintprogress now can disable printing anything and return ETA at thecall0.6.3 (2023-04-01)lognflow class does all logviewer does. Maybe it is time to remove logviewer.0.6.4 (2023-04-06)Better documentation and examples for readmeget_var is added to lognflow to get buffered variables logged by log_var0.6.5 (2023-04-26)Fixed a bug in the docs to allow sphinx compile it.log_var will log only the valid time stamps.added end keyword argument to log_text0.6.6 (2023-04-27)Better documentationadded tifffile imread to logviewer and imwrite to lognflow0.6.7 (2023-04-27)A bug in tifffile support was fixed0.6.8 (2023-05-04)Fixing readme for PyPI.removed marker from log_plot. user marker and linestyle keyword arguments.printprogress returns proper ETA every time if print_function is set to None::0.7.0 (2023-05-15)logviewer returns data by log_sigle if the full name is mentioned.0.7.1 (2023-05-22)printprogress supports lognflow.bugs fixed in lognflow.For now I guess lognflow and logviewer could be separate.0.7.2 (2023-05-25)bug fixed in logviewertext_to_object added to logviewer to read dict or list logged via log_singletest pass for logviewer including the test for text_to_object0.7.3 (2023-06-01)bug fixed in logviewer in the use of suffix in get_stack_of_fileslog_imshow takes colorbar and remove_axis_ticks flags.every lognflow instance has a logviewer pointing to its log_dir called logged.0.7.4 (2023-06-26)critical bug fixed in log_imshow0.7.5 (2023-06-27)Added complex numbers to log_imshow0.7.6 (2023-07-17)printprogress can handle up to 99 dayslog_text takes any save_asIf variable name has escape key is alrightIf variable name is splitable, we replace them with _0.8.0 (2023-07-25)logger.save and savez are set to be identical to log_single.logged.load is set to be identical to get_single.utils.py is added to contain all misc functions.replace_all added to utils0.8.1 (2023-07-26)a bug fixed in log_var0.8.2 (2023-08-02)the word save_as is now replaced with suffix as is in pathliball loggers can take the suffix as the extension in the parameter_name0.8.3 (2023-08-02)critical bug fixed in log_var to support v0.8.20.8.4 (2023-08-03)variable names that are pecular will always be fixed first.suffix can be read form the file name.time_tag will always accompany file name unless stated otherwised.0.8.5 (2023-08-04)Some functions can go to utils and be mentioned in theinita bug was fixed in printprogress.0.8.6 (2023-08-04)plt_utils was not added tp 0.8.50.9.0 (2023-08-09)copy() is now possible from a file or a variable name into anotherdefault suffix in get_flist is *logviewer.get_stack_of_files is only useful for reading data.more tests are added.moved multichannel_to_frame to utils0.9.1 (2023-08-25)bug removed from plt_utils numbers_as_images_4D.bug removed from printprogress when number of steps is very small.0.10.0 (2023-09-01)I added multiprocessor to lognflowbug fixed in logviewer0.10.1 (2023-09-12)multi_channel_by_subplots bug fixed for non-square shapedefault colormap is viridis everywheremultiprocessor heavily debugged and made a lot easier to usebetter tests added for multiprocessor0.10.2 (2023-10-04)printprogress can be used as an iterator, test addedtime_tag is False by default for copy()to log MATLAB files, input must be a dictionary.bug fixed in get_flist to return dirs only as wellall new features for Python 3.10 onwards are removed.0.10.3 (2023-10-09)multiprocessor handles errors with maximum speed as all processes share error_eventmultichannel_plots assume fitrst fimension is the channels not the lastprintprogress as iterator does not take the firstnextas a ticklog_imshow takes meaningful input sizes to make one framelog_imshow_by_subplots can put images in different placeslog_imshow_series is the new name of the log_canvasif there are multiple images the shape should be n_f x n_r x n_ctests are added for all functionstests for lognflow.utils is added0.10.4 (2023-10-12)get_flist returns whatever search pattern means for .globplt_tight_layout is removed and replaced by bboxYou can get name from file when file is within the log_dir root0.10.5 (2023-10-18)Added new files for readthedocscopy() checks for proper use of argumentscallreturns fpathloopprocessor is added0.10.6 (2023-10-19)bugs fixed in multiprocessor and loopprocessortests added0.10.7 (2023-11-01)multiprocessor_gen is a generator that yields the list of arrived resultsget_flist and thus get_single do not put asterisks on their own.0.10.8 (2023-11-03)a bug fixed in get_flist0.10.9 (2023-12-01)a bug fixed in name generator when suffix is givencopy returns destination pathexists_ok can be given to make the log_dir of lognflowadded get_namelist and its testbug fixed in plt_utilsplt_imshow added to plt_utils0.10.10 (2024-01-30)rgb2hsv is addedplt_imshow supports complex color map and is bug freeadded printprogress to loopprocessor0.11.0 (2024-02-25)is_builtin_collection is added for multiprocessor concatenationsetting time_tag to 'counter' or 'counter&time' will count filenames instead of timeplt_violinplot was addedplt_imhist is added