Package: pyml-toolbox
PyML_ToolboxPyML_Toolbox (Python Machine Learning Toolbox) is a Python package designed to assist data scientists and Jupyter notebook users in securely handling credentials, database connections, and file operations. This toolbox offers a collection of classes and functions that simplify various tasks related to data processing and management.InstallationYou can install the PyML Toolbox package using pip:pipinstallpyml_toolboxClasses and FunctionsEngine ClassTheEngineclass enables seamless management of stored logins, database engine configurations, and query executions.Example Usage:frompyml_toolboximportEngine# Initialize an Engine object with optional parametersE=Engine(dir_engine='/file/path',is_encode=True,log_level=20)# Add a new login credential for a database source, By storing the credentials using the Engine class once, there is no need to expose them within Jupyter notebooks any more.E.set_login(login='snowflake',user='name',password='pw')# Set the URL template for a database driverE.set_driver('snowflake',url='{driver}://{user}:{password}@{host}/{database}/{schema}?role={role}&warehouse={warehouse}')# Add a new engine parameter for a database sourcedict_snowflake={'driver':'snowflake','login':'snowflake','host':'snowflake.host','database':'db','schema':'sc','warehouse':'wh','role':'user_role'}E.set_engine('snowflake',**dict_snowflake)# Execute a query in a database engine and retrieve the result as a DataFramequery='''SELECT 1'''df=E.from_sql(query,'snowflake')ConfHandler ClassTheConfHandlerclass is designed for handling configuration data stored in a dictionary and saving it in a JSON file.Example Usage:frompyml_toolboximportConfHandler# Initialize a ConfHandler object with a configuration fileconf_handler=ConfHandler(file_conf='conf1.json',log_level=20)# Set initial configuration dataconf_handler.set_conf({'v1':1,'v2':{'a':1,'b':3}})# Update a key in the configuration dataconf_handler.update_key('v2',{'b':2})# Get the value of a key in the configuration datavalue=conf_handler.get_key('v2')PathHandler ClassThePathHandlerclass simplifies file path handling, including setting and getting components of a file path, adding additional directories, checking path existence, and more.Example Usage:frompyml_toolboximportPathHandler# Initialize a PathHandler object with file componentspath_handler=PathHandler(file_dir='/tmp/test',file_name='script',file_suffix='sh')# Add additional directories to the pathpath_handler.add_path('subdir1')# Check if the file's directory exists and create it if notifnotpath_handler.is_path():path_handler.create_path()JsonFile ClassTheJsonFileclass provides functions for performing JSON file operations.Example Usage:frompyml_toolboximportJsonFile# Initialize a JsonFile objectjson_file=JsonFile('data.json')# Write a dictionary to the JSON filedata={'name':'John','age':30}json_file.write(data)# Read the JSON file and retrieve the dictionaryread_data=json_file.read()CsvFile ClassTheCsvFileclass simplifies reading and writing CSV files using Pandas DataFrame.Example Usage:frompyml_toolboximportCsvFileimportpandasaspd# Initialize a CsvFile objectcsv_file=CsvFile('data.csv')# Write DataFrame to CSV filedf=pd.DataFrame({'name':['John','Jane'],'age':[30,25]})csv_file.write(df)# Read CSV file into a Pandas DataFrameread_df=csv_file.read()ParquetFile ClassTheParquetFileclass facilitates reading and writing Parquet files using Pandas DataFrame.Example Usage:frompyml_toolboximportParquetFileimportpandasaspd# Initialize a ParquetFile objectparquet_file=ParquetFile('data.parquet')# Write DataFrame to Parquet filedf=pd.DataFrame({'name':['John','Jane'],'age':[30,25]})parquet_file.write(df)# Read Parquet file into a Pandas DataFrameread_df=parquet_file.read()init_logger FunctionTheinit_loggerfunction initializes a logger within a Jupyter notebook and returns a logger with the specified name.Example Usage:frompyml_toolboximportinit_logger# For example, initiate a logger inside a classself.logger=init_logger(self.__class__.__name__,level=log_level)LicensePyML_Toolbox is licensed under the GNU General Public License v3 (GPLv3).TODOsAdd a simple model registry which stores time, data, hyperparameters, and other relevant features of the model training process.