Source control for data scientists
This chapter covers In the last chapter, we introduced several key software engineering concepts that will improve your life as a data scientist. These included: Source control Exception handling Putting a model into production Object-oriented programming (OOP) Automated testing Scale In this chapter, we’re going to delve deeper into the first of these - namely, . source control Source control (also called ) is basically a way of tracking changes to a codebase. version control As the number and size of codebases has grown indescribably over the years, the need for monitoring code changes, and making it easier for various developers to collaborate is absolutely crucial. Because software engineering has existed longer than modern data science, source control has been a software engineering concept longer than a data science one. However, as we’ll demonstrate in this chapter, source control is an important tool to learn for any data scientist. Before we delve into using source control for data science, however, let’s discuss how Source control for data scientists What is source control Why do data scientists need to know source control What is git What are the main git commands you need to know as a data scientist



—> source control Source control (also called version control ) : is basically a way of tracking changes to a codease 
—> As the number and size of codebases has grown indescribably over the years , the need  for monitoring code changes ,
—> make easy to collaborate various developers to collaborate is absolutely 



—> version contral come into the umbrealla of applying software engineering to data science 


Next, let’s explore how source control will help you in your projects. Going back to the customer churn example from the first chapter, suppose you and a colleague are working on a data science project together to predict whether a customer will churn. To make this concrete, we will use the customer churn dataset available from Kaggle here: www.kaggle.com/competitions/customer-churn-prediction-2020/data. This dataset involves predicting whether a customer from a Telecom company will churn. The workflow of this project can be structured similarly to the data science life cycle that we discussed in the first chapter. As a refresher, we’ve repeated the chapter one diagram showing the combined data pipeline/ML pipeline view in Figure 2.1

In the following steps, we tie the pipeline components to our specific Kaggle dataset: Gathering data Fetch the data needed to build the customer churn model. In the Kaggle dataset we’ll be using, this includes information like length of account (account age), number of day calls, area code, etc.

In the following steps, we tie the pipeline components to our specific Kaggle dataset: Gathering data Fetch the data needed to build the customer churn model. In the Kaggle dataset we’ll be using, this includes information like length of account (account age), number of day calls, area code, etc. Exploratory data analysis (EDA) / data validation Analyzing the data for patterns, distributions, correlations, missing values, etc. For example, what’s the proportion of churn to non-churn? What’s the association with total day minutes used (total_day_minutes) and churn? Etc. Data cleaning Handle issues, such as missing values, outliers, dirty data, etc. Feature engineering Create new features for the models you both will build. For example, you could create a new feature based on the average total day charge for the state the user resides in. Model training Develop models, such as logistic regression or random forest to predict churn. Model evaluation. ©Manning Publications Co. To comment go to liveBook https://livebook.manning.com/book/software-engineering-for-data-scientists/discussion 26 Evaluate the performance of the models. Since churn vs. non-churn is a classification problem, we might use metrics like precision or recall here. Model deployment Make the finalized model accessible to others. For this case, that might mean prioritizing individuals predicted to churn for marketing or lower price incentives.

Each of these steps might involve collaboration between you and your colleague. Consider a few example scenarios: When gathering data, you might write code to extract data from the last year. Your colleague, wishing to get more data, modifies the codebase to extract data from the last three years. Your colleague adds code that creates several visualizations of the data, like the distribution of daytime calls. You find a bug in your colleague’s code and want to correct it, so you modify the shared codebase. The two of you are working on cleaning the data. One colleague adds code to replace missing values and handle outliers. The other one realizes that one of the fields has a mix of numeric and string values, and writes code to clean this issue. Both sets of changes go into the same file. One of you works on developing an logistic regression model, while the other wants to try out a random forest. But you both want to share code with each other and potentially make modifications (like use different parameters or features) Source control makes these types of collaborations much easier and trackable. Next, let’s dive into how source control would help you and your colleague in this situation, or in general for any data science project you may work on.

Q- oky now i just clone this repo 

https://github.com/AmirAbda/Mlops_Customer_Churn_Prediction-.git

Open it in vsCode 

And create a python venv env like this 
PS C:\Users\PCS\Desktop\mlops\Costumer_chun\Mlops_Customer_Churn_Prediction->
PS C:\Users\PCS\Desktop\mlops\Costumer_chun\Mlops_Customer_Churn_Prediction-> python -m venv venv
PS C:\Users\PCS\Desktop\mlops\Costumer_chun\Mlops_Customer_Churn_Prediction-> venv\Scripts\activate
(venv) PS C:\Users\PCS\Desktop\mlops\Costumer_chun\Mlops_Customer_Churn_Prediction-> pip install pandas numpy matplotlib seaborn
Collecting pandas
  Obtaining dependency information for pandas from https://files.pythonhosted.org/packages/ed/8c/87ddf1fcb55d11f9f847e3c69bb1c6f8e46e2f40ab1a2d2abadb2401b007/pandas-2.2.3-cp311-cp311-win_amd64.whl.metadata
  Using cached pandas-2.2.3-cp311-cp311-win_amd64.whl.metadata (19 kB)
Collecting numpy
  Obtaining dependency information for numpy from https://files.pythonhosted.org/packages/d4/96/450054662295125af861d48d2c4bc081dadcf1974a879b2104613157aa62/numpy-2.1.2-cp311-cp311-win_amd64.whl.metadata
  Using cached numpy-2.1.2-cp311-cp311-win_amd64.whl.metadata (59 kB)
Collecting matplotlib
  Obtaining dependency information for matplotlib from https://files.pythonhosted.org/packages/8b/ce/15b0bb2fb29b3d46211d8ca740b96b5232499fc49200b58b8d571292c9a6/matplotlib-3.9.2-cp311-cp311-win_amd64.whl.metadata
  Using cached matplotlib-3.9.2-cp311-cp311-win_amd64.whl.metadata (11 kB)
Collecting seaborn
  Obtaining dependency information for seaborn from https://files.pythonhosted.org/packages/83/11/00d3c3dfc25ad54e731d91449895a79e4bf2384dc3ac01809010ba88f6d5/seaborn-0.13.2-py3-none-any.whl.metadata
  Using cached seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB)
Collecting python-dateutil>=2.8.2 (from pandas)
  Obtaining dependency information for python-dateutil>=2.8.2 from https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas)
  Obtaining dependency information for pytz>=2020.1 from https://files.pythonhosted.org/packages/11/c3/005fcca25ce078d2cc29fd559379817424e94885510568bc1bc53d7d5846/pytz-2024.2-py2.py3-none-any.whl.metadata     
  Using cached pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas)
  Obtaining dependency information for tzdata>=2022.7 from https://files.pythonhosted.org/packages/a6/ab/7e5f53c3b9d14972843a647d8d7a853969a58aecc7559cb3267302c94774/tzdata-2024.2-py2.py3-none-any.whl.metadata 
  Using cached tzdata-2024.2-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting contourpy>=1.0.1 (from matplotlib)
  Obtaining dependency information for contourpy>=1.0.1 from https://files.pythonhosted.org/packages/8d/2f/804f02ff30a7fae21f98198828d0857439ec4c91a96e20cf2d6c49372966/contourpy-1.3.0-cp311-cp311-win_amd64.whl.metadata
  Using cached contourpy-1.3.0-cp311-cp311-win_amd64.whl.metadata (5.4 kB)
Collecting cycler>=0.10 (from matplotlib)
  Obtaining dependency information for cycler>=0.10 from https://files.pythonhosted.org/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl.metadata       
  Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib)
  Obtaining dependency information for fonttools>=4.22.0 from https://files.pythonhosted.org/packages/63/f1/3a081cd047d83b5966cb0d7ef3fea929ee6eddeb94d8fbfdb2a19bd60cc7/fonttools-4.54.1-cp311-cp311-win_amd64.whl.metadata
  Using cached fonttools-4.54.1-cp311-cp311-win_amd64.whl.metadata (167 kB)
Collecting kiwisolver>=1.3.1 (from matplotlib)
  Obtaining dependency information for kiwisolver>=1.3.1 from https://files.pythonhosted.org/packages/a1/65/d43e9a20aabcf2e798ad1aff6c143ae3a42cf506754bcb6a7ed8259c8425/kiwisolver-1.4.7-cp311-cp311-win_amd64.whl.metadata
  Using cached kiwisolver-1.4.7-cp311-cp311-win_amd64.whl.metadata (6.4 kB)
Collecting packaging>=20.0 (from matplotlib)
  Obtaining dependency information for packaging>=20.0 from https://files.pythonhosted.org/packages/08/aa/cc0199a5f0ad350994d660967a8efb233fe0416e4639146c089643407ce6/packaging-24.1-py3-none-any.whl.metadata   
  Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting pillow>=8 (from matplotlib)
  Obtaining dependency information for pillow>=8 from https://files.pythonhosted.org/packages/c1/d0/5866318eec2b801cdb8c82abf190c8343d8a1cd8bf5a0c17444a6f268291/pillow-10.4.0-cp311-cp311-win_amd64.whl.metadata 
  Using cached pillow-10.4.0-cp311-cp311-win_amd64.whl.metadata (9.3 kB)
Collecting pyparsing>=2.3.1 (from matplotlib)
  Obtaining dependency information for pyparsing>=2.3.1 from https://files.pythonhosted.org/packages/e5/0c/0e3c05b1c87bb6a1c76d281b0f35e78d2d80ac91b5f8f524cebf77f51049/pyparsing-3.1.4-py3-none-any.whl.metadata 
  Using cached pyparsing-3.1.4-py3-none-any.whl.metadata (5.1 kB)
Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas)
  Obtaining dependency information for six>=1.5 from https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl.metadata
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Using cached pandas-2.2.3-cp311-cp311-win_amd64.whl (11.6 MB)
Using cached numpy-2.1.2-cp311-cp311-win_amd64.whl (12.9 MB)
six, pyparsing, pillow, packaging, numpy, kiwisolver, fonttools, cycler, python-dateutil, contourpy, pandas, matplotlib, seaborn
Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.54.1 kiwisolver-1.4.7 matplotlib-3.9.2 numpy-2.1.2 packaging-24.1 pandas-2.2.3 pillow-10.4.0 pyparsing-3.1.4 python-dateutil-2.9.0.post0 pytz-2024.2 seaborn-0.13.2 six-1.16.0 tzdata-2024.2

[notice] A new release of pip is available: 23.2.1 -> 24.2
[notice] To update, run: python.exe -m pip install --upgrade pip
(venv) PS C:\Users\PCS\Desktop\mlops\Costumer_chun\Mlops_Customer_Churn_Prediction->


Q- i just add it a folder called “data” 
And add the test.csv and train.csv to it 

Next i want to learn my fist gitworklow to push this change to my remote server witout adding my venv wich my not necessary just he requirement.txt file $

Q- give me step by step workflow to it oky