Package: datadict-toolbox
Dictionary_ToolBox (v1.0.1)Dictionary_ToolBox is a python package containing classes for building a tabular, multidimensional cube data dictionary and SQL select queries using OpenAI's GPT templates.This package processes.xmlafiles generated by MS SQL SERVER software.Extracts SQL queries from.dtsxfiles.Generates a partial data dictionary from a SQL select query.InstallationYou can installdatadict_toolboxusingpipfrom the Python Package Index (PyPI).pipinstalldatadict_toolboxThe package is hosted on PyPI and can be found athttps://pypi.org/project/datadict-toolbox/. You can visit the link for more information about the package and its available versions.SetupBefore you start using DataDict Toolbox, you need to set up a suitable Python environment. We recommend using a virtual environment (venv). Here's how you can set this up:Install a Python  Interpreter (3.9) minimum. You can download it from the official Python website. Make sure to allow the installer to set the PATH variables for you.Check your Python version by runningpython --versionorpython3 --versionfrom the command line.Once you have Python, you can create a virtual environment or it's automatically generated in IDE as Pycharm. To do this, run the following command in the root directory of the project:python-mvenvvenvThis command creates a new directory namedvenvin your project. This directory will contain the Python executable files and a copy of the pip library which you can use to install other packages within this environment.To start using this environment, you have to activate it.On Unix or MacOS, run:sourcevenv/bin/activateOn Windows, run:venv\Scripts\activateOnce the virtual environment is activated, you can install the necessary dependencies. To do this, run the following command:pipinstall-rrequirements.txtNow you should be all set! Remember to activate the venv environment every time you work on the project.Finally, ensure you have an OpenAI API key and organization ID. The API key is necessary for making requests to the OpenAI service. To use them safely you must write them in a.envfile and use thepython-dotenvPython Package to get the key and id.UsageImporting the packageThe easiest way to use the package is to copy the following codes in yourmain.pyfile to use the different classes :Class: ExtractorTabularCubeCatalogCreate a tabular cube data dictionary from one .xmla filefromdatadict_toolboximportExtractorTabularCubeCatalogfile_path="paste the path of your .xmla file"etcc=ExtractorTabularCubeCatalog(file_path)# Generate the data dictionary at the Instance of the classprint(etcc.cube_struct)# Display the python dictionary that contain your data dictionaryetcc.save()# By default it will create a "catalog_name_tabular_nbrtable.xslx" in a "excel_result/" folder automatically created in the root path of your running .py fileCreate several tabular cube data dictionary from a folder of .xmla filesimportosfromdatadict_toolboximportExtractorTabularCubeCatalogfolder_path="paste the path of the folder containing your .xmla files"files_path=[os.path.join(folder_path,f)forfinos.listdir(folder_path)ifos.path.isfile(os.path.join(folder_path,f))]print("Nbr fichier: ",len(files_path))forfile_pathinfiles_path:etcc=ExtractorTabularCubeCatalog(file_path)# Generate the data dictionary at the Instance of the classprint(etcc.cube_struct)# Display the python dictionary that contain your data dictionaryetcc.save()# By default it will create a "catalog_name_tabular_nbrtable.xslx" in a "excel_result/" folder automatically created in the root path of your running .py fileYou can change the save path and the filename of the.xlsxfile by changing thesave_pathandfilenameparameters.Class: ExtractorMultidimCubeCatalogCreate a multidimensional cube data dictionary from one .xmla filefromdatadict_toolboximportExtractorMultidimCubeCatalogfile_path="paste the path of your .xmla file"emcc=ExtractorMultidimCubeCatalog(file_path)# Generate the data dictionary at the Instance of the classprint(emcc.cube_struct)# Display the python dictionary that contain your data dictionaryemcc.save()# By default it will create a "catalog_name_tabular_nbrtable.xslx" in a "excel_result/" folder automatically created in the root path of your running .py fileCreate several multidimensional cube data dictionary from a folder of .xmla filesimportosfromdatadict_toolboximportExtractorMultidimCubeCatalogfolder_path="paste the path of the folder containing your .xmla files"files_path=[os.path.join(folder_path,f)forfinos.listdir(folder_path)ifos.path.isfile(os.path.join(folder_path,f))]print("Nbr fichier: ",len(files_path))forfile_pathinfiles_path:emcc=ExtractorMultidimCubeCatalog(file_path)# Generate the data dictionary at the Instance of the classprint(emcc.cube_struct)# Display the python dictionary that contain your data dictionaryemcc.save()# By default it will create a "catalog_name_tabular_nbrtable.xslx" in a "excel_result/" folder automatically created in the root path of your running .py fileYou can change the save path and the filename of the.xlsxfile by changing thesave_pathandfilenameparameters.Class: SelectGPTDeduceimportosfromdotenvimportload_dotenvfromdatadict_toolboximportSelectGPTDeduceload_dotenv("path of your .env file relative to your root path")# load .env fileopenai_api_key=os.getenv("OPENAI_API_KEY")# get OPENAI_KEY value from .env file{}openai_org_id=os.getenv("OPENAI_ORG_ID")select_query="paste your SQL select query here"deduce=SelectGPTDeduce(openai_org_id,openai_api_key,select_query)print(deduce.select_data_dict)# Display the python dictionary that contain your data dictionarydeduce.save()Thesave()method is not configurable in this version.File: extract_from_dtsx.pyThe most usefull function is theextract_erp_querywhich extract the SQL query from a.dtsxfile.importosfromdatadict_toolboximportextract_erp_queryfolder_path="paste the path of the folder containing your .xmla files"files_path=[os.path.join(folder_path,f)forfinos.listdir(folder_path)ifos.path.isfile(os.path.join(folder_path,f))]queries=extract_erp_query(files_path)['SQL_QUERY']This function takes a list as argument and outputs a dictionary of 2 keyDEST_TABLEandSQL_QUERY.
You can fill in theSelectGPTDeduceclass the outputs of theextract_erp_query.CLINo command for this packageConfigurationif you don't have a.envfile create it and put 2 important variablesOPENAI_API_KEY="your api key"
OPENAI_ORG_ID="your organization id"The assessment aims to provide comprehensive information that can help a new developer understand the purpose and functionality of the code, as well as areas that could potentially be refactored or optimized.GoalThis package was done for a internship project. It will probably not be maintained in the future.LicenseThis project is licensed under the MIT License. See the LICENSE file for details.