Write FULL R Code for this:

1. Visualization
In linear regression, data distribution and relationship are critical. In particular, histogram is to examine if the observations are skewed and scatter plot is to examine if the relationship is linear. Examine histograms and scatter plots for the variables of your interests. If there are some concerns, try different wrangling to mitigate the concerns.
Look at correlations between variables and try to identify sources of concern. In R, you can use cor function. Pay particular attention to the correlation for total and P15 through P55. Do these correlations make sense to you?
Create new variables by dividing EARN, P15, P25, P35, P45, P55, COMP, NCOMP, and NREST by total. These new variables are per capita. Examine correlations with the transformed. Do these correlations make sense to you?
If you have more ideas for a better model, please feel free to create new variables. Examine them visually before using them.

2-1. Building Models (Part a)
Consider all observations in CroqPainFix.
Michel’s first model is shown in Table 6.27 on page 305 with some concerns. Try to improve the model that does not violate any of the basic assumptions of regression but has good predictive power. In short, you should carefully choose the explanatory variables. What are the explanatory variables of your choice?
When you run regressions, be sure to check multicollinearity.
If you want, you can try different variable selection algorithms. However, be critical and make a decision about the final model to recommend for Croq’Pain.


2-2. Validation by Testing (Part b)
In 2-1, you build a model using all observations in CroqPainFix. Michel wants to validate its usefulness (or accuracy) with existing data. Split CroqPainFix into the following two data sets.
CroqPainFix_50: The 50 stores opened up prior to 1994, i.e., STOR<=50.
CroqPainFix_10: The 10 stores opened in the first half of 1994, i.e., STOR>50.
Using CroqPainFix_50, re-estimate your model. That is, keep your explanatory variables from 2-1 and re-estimate their coefficients with the data set of 50 stores.
With this newly estimated model, predict the performances of the 10 stores in CroqPainFix_10. Using Croq’Pain’s performance ratio target of 26%, which of the ten stores would you have opened in 1994 according to your model?

3. Prediction (Part c)
As you validated the model (i.e., a selection of explanatory variables), let’s use the most complete estimates using CroqPainFix. With this complete model, you need to recommend which new locations Croq’Pain should open its stores among 10 locations in Table 6.28 or CroqPain_new.rda. Which locations would you recommend?

CroqPain_new:
The EMPL column contains NA values.
The STOR column should be converted to a factor, and the EMPL column should be converted to an integer (after handling missing values).

CroqPaiFix:
The STOR column should be converted to a factor. Numeric columns should be of type numeric or integer

Notes:
1. Data Exploration and Visualization:
1.1. Data Distribution and Relationship Examination:
Histogram: Use histograms to examine the distribution of key variables like EARN, P15, P25, P35, P45, P55, COMP, NCOMP, NREST, and total. This will help identify any skewness or unusual distributions.
Scatter Plot: Use scatter plots to examine the relationship between EARN and other variables like SIZE, EMPL, INC, PRICE, CLI, etc. This will help identify any linear or non-linear relationships.
Print answers and plot with ggplot2 if needed

Libraries to Use:
ggplot2: For creating histograms and scatter plots.
dplyr: For data manipulation and wrangling.

1.2. Correlation Analysis
Correlation Matrix: Use the cor() function to create a correlation matrix for all variables. Focus on the correlations between total and P15 through P55.
Interpretation: Analyze the correlation matrix to identify any strong correlations or multicollinearity issues.
Print answers and plot with ggplot2 if needed

1.3. Variable Transformation:
Per Capita Variables: Create new variables by dividing EARN, P15, P25, P35, P45, P55, COMP, NCOMP, and NREST by total.
Correlation with Transformed Variables: Re-examine the correlation matrix with these new per capita variables.
Print answers and plot with ggplot2 if needed

1.4. Additional Variable Creation:
Ideas for New Variables: If you have any ideas for new variables that might improve the model, create them and examine them visually using histograms and scatter plots.

2. Building and Validating Models
2.1. Building Models (Part a)
Initial Model: Start with Michel’s initial model as a baseline.
Model Improvement: Improve the model by carefully selecting explanatory variables that do not violate basic regression assumptions (e.g., linearity, homoscedasticity, independence, normality of residuals).
Explanatory Variables: Choose explanatory variables based on their significance, correlation with the response variable, and absence of multicollinearity.
Print answers and plot with ggplot2 if needed

Multicollinearity Check: Use the car package to check for multicollinearity using the vif() function.
Print answers and plot with ggplot2 if needed

Variable Selection Algorithms: Optionally, use algorithms like stepwise regression (stepAIC from the MASS package) or LASSO regression (glmnet package) for variable selection.
Print answers and plot with ggplot2 if needed

Final Model: Decide on the final model to recommend for Croq’Pain.
Print answers and plot with ggplot2 if needed

2.2. Validation by Testing (Part b)
Data Splitting: Split CroqPainFix into CroqPainFix_50 (stores opened before 1994) and CroqPainFix_10 (stores opened in the first half of 1994).
Model Re-estimation: Re-estimate the model using CroqPainFix_50 and keep the same explanatory variables.
Prediction: Use the re-estimated model to predict the performances of the 10 stores in CroqPainFix_10.
Print answers and plot with ggplot2 if needed
Performance Ratio Target: Compare the predicted performances against Croq’Pain’s performance ratio target of 26% to decide which stores to open.

Libraries to Use:
stats: For linear regression.
car: For checking multicollinearity.
MASS: For stepwise regression.
glmnet: For LASSO regression.

3. Prediction (Part c)
3.1. Complete Model Estimation

Final Model: Use the most complete estimates from CroqPainFix to build the final model.
Prediction for New Locations: Use the final model to predict the performances of the 10 new locations in CroqPain_new.
Print answers and plot with ggplot2 if needed
Recommendation: Based on the predictions, recommend which new locations Croq’Pain should open its stores.

Libraries to Use:
stats: For linear regression and prediction.

n to check the data types of each column.

Convert Data Types: Ensure that columns like STOR, EMPL, and other categorical variables are of type factor. Numeric columns should be of type numeric or integer.

> colnames(CroqPain_new)
 [1] "STOR"        "K"           "SIZE"       
 [4] "EMPL"        "total"       "P15"        
 [7] "P25"         "P35"         "P45"        
[10] "P55"         "INC"         "COMP"       
[13] "NCOMP"       "NREST"       "PRICE"      
[16] "CLI"         "P15_total"   "P25_total"  
[19] "P35_total"   "P45_total"   "P55_total"  
[22] "COMP_total"  "NCOMP_total" "NREST_total"
> colnames(CroqPainFix)
 [1] "STOR"             "EARN"            
 [3] "K"                "SIZE"            
 [5] "EMPL"             "total"           
 [7] "P15"              "P25"             
 [9] "P35"              "P45"             
[11] "P55"              "INC"             
[13] "COMP"             "NCOMP"           
[15] "NREST"            "PRICE"           
[17] "CLI"              "EARN_per_capita" 
[19] "P15_per_capita"   "P25_per_capita"  
[21] "P35_per_capita"   "P45_per_capita"  
[23] "P55_per_capita"   "COMP_per_capita" 
[25] "NCOMP_per_capita" "NREST_per_capita"

Issues:
Data Limitations:
Missing Values: The "EMPL" (number of employees) data is missing for the potential new store locations. [2-9] As discussed in our conversation, how you handle these missing values will affect the model's ability to predict the performance of these specific locations.
Unknown Problematic Variables: The sources mention Michel's concerns about certain variables being unsuitable for prediction because they are either unknown in advance or lack objective justification. [1] You'll need to identify these variables through careful examination of the variable list and determine how to address their inclusion.
Multicollinearity: The sources and our conversation highlight the high correlations between "total" population and the age group variables (P15-P55) as a potential source of multicollinearity. [10, 11]
Addressing Multicollinearity: Creating per capita variables, as explored in our conversation, is a recommended strategy to mitigate this issue. However, carefully monitor the impact of this transformation on the model's performance and interpretation.
Missing Values: Both sources show that the "EMPL" (number of employees) column is blank for the ten potential new store locations (Table 6.28 and the str(CroqPain_new) output).
orrelations between 'total' and Age Group Variables: Source highlights potentially problematic correlations between "total" population and the age group variables (P15-P55). This concern was also a focus in our previous conversation.
Addressing the Issue: Creating per capita variables (e.g., P15/total, P25/total, etc.) is suggested to mitigate multicollinearity stemming from these correlations. This transformation aims to capture the proportion of each age group within the total population.
Michel's Concerns about "Parameter Choice" and "Homemade Parameters": Source reveals Michel's doubts about the suitability of some variables in the initial model.
Further Investigation: The sources don't explicitly identify these problematic variables. You'll need to carefully examine the variables listed in Table 6.25 and consider:
Data Availability: Can values for these variables be reliably obtained before a store opens, which is crucial for location selection?
Objectivity and Relevance: Are any variables based on subjective expert opinions or lack a clear, justifiable connection to store profitability? Such variables might need to be refined or replaced.
The sources don't explicitly confirm if the data is already clean.
Missing values in the "EMPL" variable require attention, especially for predicting the performance of new locations.
The high correlations between "total" and the age group variables warrant creating per capita variables.
You'll need to investigate and address Michel's concerns about potentially problematic variables.
Exploring the data for outliers and distribution issues using visualizations is essential for building a reliable regression model.