Package: housing-package
Median housing value predictionThe housing data can be downloaded fromhttps://raw.githubusercontent.com/ageron/handson-ml/master/. The script has codes to download the data. We have modelled the median house value on given housing data.The following techniques have been used:Linear regressionDecision TreeRandom ForestSteps performedWe prepar and clean the data. We check and imput for missing values.Features are generated and the variables are checked for correlation.Multiple sampling techinuqies are evaluated. The data set is split into train and test.All the above said modelling techniques are tried and evaluated. The final metric used to evaluate is mean squared error.To run the codeWe have to create an environment from env.yml file which is attached in this repository .Command to create environment -conda env create -f env.ymlCommand tp see newly created environment -conda env listCommand to activate the environment -conda activate mle-devNow execute the python script .To execute the python scriptWe have to open the mlflow ui . my mlflow listening at -http://127.0.0.1:5000you can check your mlflow uri bymlflow uicommand .If your mlflow is listening at different uri then need to change the mlflowrun.py line 8 remote_server_uri .Now run the python script .command to run the scripts -python scripts/mlflowrun.pyORWe can run individual file from src -> py_packages folderpython src/py_packages/ingest.py <--path , --log_level, --log_path --no_console_log>
python src/py_packages/train.py <--train_folder,--output_folder,--log_level,--log_path,--no_consoler_log>
python src/py_packages/score.py <--model-folder,--data_folder,--log_level,--log_path,--no_consoler_log>ingest_data : To download the data and split into train and test (arguments can be passed to give download path)While executing the script to terminal , data will get stored as per the default settingsArguments that can be passed : <--path , --log_level, --log_path --no_console_log>--path :: Specify the path for the training dataset--log_level :: Specify the level of log required--log_path :: Specify the location where logs to be created (If not specified no logs will be created)--no_console_log :: Specify the log to consoletrain : To fit and train different models based on the training data providedArgumentd that can be passed : <--train_folder,--output_folder,--log_level,--log_path,--no_consoler_log>--train_folder :: Specify path with file name where the train data is present.--output_folder :: Specify the location where the trained models pkl files to save.--log_level :: Specify the level of log required--log_path :: Specify the location where logs to be created (If not specified no logs will be created)--no_console_log :: Specify the log to consolescore : To check the scores of model on the new data passed.Argumentd that can be passed : <--model-folder,--output-folder,--log_level,--log_path,--no_consoler_log>--model_folder :: Specify the location where the models pkl file saved.--data_folder :: Specify the location where test datasets are present.--log_level :: Specify the level of log required--log_path :: Specify the location where logs to be created (If not specified no logs will be created)--no_console_log :: Specify the log to consoleTo execute the test python scriptCommand to execute the test script of data ingestion module-python test/test_data_ingestion.pyCommand to execute the test script of train module -python test/test_train.pyCommand to execute the test script of score module -python test/test_score.pyCommand to execute the test script to check the installation -python test/test_installation.pyORWe can directly run all 4 test script using -pytestcommand .To create html in docs using sphinxTo document the housing package , We need to create a folder named 'docs'.Command to go to docs folder -cd docsCommand to generates a complete documentation directory along with a make.bat file, which will be used later to generate HTML -sphinx-quickstartAfter running the command, accept the defaults.Now ,go to conf.py file. Add 'os.path.abspath('..')' above project information. Here, it will tell sphinx that the code is residing outside of the current docs folder.Next, go to the extensions part and add the extensions - ["sphinx.ext.autodoc", "sphinx.ext.viewcode", "sphinx.ext.napoleon"]Lastly, go to themes and replace ‘alabaster’ with ‘sphinx_rtd_theme’.Now , go to parent folder -cd ..Run command -sphinx-apidoc -o docs src/housing_package/It will generate housing_package.rst and modules.rst files .Now, include the generated modules.rst file in your index.rstGo inside the docs folder and run the command -make htmlWe can see HTML files generated inside _build folder in docs .