Automate Excel with Python | by KahEm Chu | Towards Data Science

Most of the time, an organization will have multiple data sources, data scientist or data analyst will have to extract and compile the data from the different data source into one Excel file before performing analysis or create the model. This could be a complex and time taking task if we do it manually.
Doing this with the most famous Python library, pandas will shorten the time, but the hard-coded Excel file which might not be favor by other domain users who will access the Excel File directly. Hence, creating an interactive Excel file, which is nicely formatted will ease the job for domain users and enlighten them.
Excel allows you to work with data in a transparent manner, meaning when an Excel file is opened, the data is visible immediately and every step of data processing is also visible. — Hong Zhou [1]
Furthermore, Excel is wonderful because we can see how the report is generated and what is the relations between the columns. The Excel file itself is self-explanatory. However, a hard-coded Excel file generated by pandas does not have these features.
So, this is where pywin32 come in. Pywin32 allowed us to access Microsoft Excel from Python. We can access all the objects, methods and properties in Excel via pywin32. (See all Excel objects, methods and properties here.)
There is no official documentation for pywin32, so I have to refer to the official documentation of Excel VBA for the objects, properties and methods available.
The objects, properties, methods and events are contained in the Excel object model (I do not work with events for now, so in this article, I will be focusing on the objects, properties and methods only). The relationship between them is illustrated in the flowchart below.
One of the objects I frequently use is the Range object. Range refers to the cell(s) in Excel Worksheet. The Range object contains Methods and Properties as shown in the figure below.
The documentation contains the basic syntax in Excel VBA. Fortunately, it is easy to translate the syntax in Excel VBA into Python. Before we go into examples, there are five items we need to understand:
Excel Object
Excel Object Methods
Excel Object Properties
Excel Constants
Excel Macro
Excel Object is similar to Python Class Module.
When we want to call a function defined in a class module, we will call the class module, followed by a dot then the function name. We need to import the class module to use its functions, but for the pywin32 module, it already imports everything, so we can directly call any Excel objects, methods and properties.
The following sections are detailed explanation of Excel objects, methods and properties with example. To provide a piece of clearer information, the example will be run with the Excel file generated with the script below.
Define the Function to Create Excel File
Define the Function to Create Excel File
def create_test_excel_file(f_path: Path, f_name: str, sheet_name: str):        filename = f_path / f_name    random.seed(365)    np.random.seed(365)    number_of_data_rows = 1000        # create list of 31 dates    dates = pd.bdate_range(datetime(2019, 9, 1), freq='1d', periods=31).tolist()data = {'Date': [random.choice(dates) for _ in range(number_of_data_rows)],            'Gender': [random.choice(['Female', 'Male']) for _ in range(number_of_data_rows)],            'Products': [random.choice(['Shirts', 'Pants', 'Caps', 'Socks']) for _ in range(number_of_data_rows)],            'Price': np.random.normal(15, 5, size=(1, number_of_data_rows))[0]}# create the dataframe and save it to Excel    pd.DataFrame(data).to_excel(filename, index=False, sheet_name=sheet_name, float_format='%.2f')
2. Define Parameters and Execute the Function
# sheet name sheet_name = 'example'  # update with sheet name from your file# file pathf_path = Path.cwd()  # file in current working directory# f_path = Path(r'c:\...\Documents')  # file located somewhere else# excel file namef_name = 'example.xlsx'create_test_excel_file(f_path, f_name, sheet_name)
Screenshot of Excel File created for your benefit.
The methods in Excel Object Model is the same as the functions in Python, it is callable and performing a task. Writing a function in python shall always end with a pair of parenthesis that holding keywords argument as shown in the Python script below, the function greet end with a pair of parenthesis that holding the argument named “name”.
# Python Functions:# create a functiondef greet(name):    print(f"Hello {name}")greet(name = "Alisa")
The script below shows how to use a method in Excel, AutoFilter by using pywin32. It is quite similar to Python function syntax, except we do not need to define the methods, and we put the object in front of the methods like we put class module name before the function we define in the class module. (The Excel File used here is the Excel File created in the section earlier.)
import pandas as pdfrom pathlib import Pathimport win32com.client as win32f_path = Path.cwd() # change to the path to the Excel filef_name = 'example.xlsx' # Excel File namefilename = f_path/f_namesheetname = 'example' # change to the name of the worksheet# create Excel objectexcel = win32.gencache.EnsureDispatch('Excel.Application')# excel can be visible or notexcel.Visible = True # open Excel Workbook   wb = excel.Workbooks.Open(filename)# create filter in Excel Worksheetwb.Sheets(sheetname).Range("A1").AutoFilter(Field = 2, Criteria1="Female", VisibleDropDown=False)
In the example above, wb.Sheets(sheetname).Range(“A1”) refers to the object, AutoFilter() is the method while Field = 2, Criteria1="Female", VisibleDropDown=False are the parameters. Field refers to the Column you want to filter. Field = 2 refers to the second column count from left, which is Column B [‘Gender’]. The result of the script is shown in the figure below, the data is filtered according to ‘Gender’, and only ‘Female’ records are showing as we set Criteria to female.
The standard syntax of writing the Excel VBA into Python is to organize the code in this way:
Object.Method(Parameter = parameter)
When you viewing the Excel VBA documentation, there will always be some example of the script as follows (The script below is modified from the example in the Excel VBA reference mentioned above).
Worksheets("Sheet1").Range("A1").AutoFilter _ Field:=2, _ Criteria1:="Female", _ VisibleDropDown:=False
There are only minor differences between the syntax of Excel VBA and Python, where Excel VBA do not hold the parameters in the parenthesis, and in Python, we have to create the Excel Object and define the Workbook path and name explicitly. Hence, it is possible and easy to translate from Excel VBA to Python.
The properties in Excel Object Model is the same as the variables in Python, it is not callable and only return assigned value. For example, the column width of a column in the Excel Sheet is properties which use to store the column width of the column. Another example, the value of a cell in an Excel Sheet. You can imagine it as a variable, which u can change its value just like you changing the value of the variable in python.
# Python Variable: # assign a value to a variablex = 5# change the value assigned to the variablex = 6
The above shows a simple example of how to update the value of a variable in Python. The script below shows how to update the value of a cell in Excel by using pywin32.
import pandas as pdfrom pathlib import Pathimport win32com.client as win32import datetimetoday = datetime.date.today()f_path = Path.cwd() # change to the path to the Excel filef_name = 'example.xlsx' # Excel File namefilename = f_path/f_namesheetname = 'example' # change to the name of the worksheet# create Excel objectexcel = win32.gencache.EnsureDispatch('Excel.Application')# excel can be visible or notexcel.Visible = True # open Excel Workbook   wb = excel.Workbooks.Open(filename)# update value in Excel Worksheetwb.Sheets(sheetname).Range("F1").Value = f"Today date: {today}"
In the example above, wb.Sheets(sheetname).Range(“F1”) refers to the object, Value is the property while f”Today date: {today}” is the value assigned to the object. In short, we put a value in a cell. As we are scripting in Python, we can apply some Pythonic script. In this case, f string is used. The figure below shows the result of the script.
The filter created from the previous script is not cleared when I run this script, so the filter is still in the file. The method to clear the filter will be explained in the section below.
The standard syntax of writing the Excel VBA into Python is to organize the code in this way:
Object.Property = value
Below is the script modified from the Excel VBA reference.
Worksheets("Sheet1").Range("F1").Value = "2021-05-13"
For properties, the syntax in Excel VBA is the same as in Python most of the time. The syntax will be slightly different when the value to be assigned is Excel constant.
Some object methods and properties only accept a list of constants for their parameters. The Excel constants normally start with “xl”. In this case, we have to import the constants when we using Python (this is not necessary when you using Excel VBA).
import pandas as pdfrom pathlib import Pathimport win32com.client as win32win32c = win32.constantsf_path = Path.cwd() # change to the path to the Excel filef_name = 'example.xlsx' # Excel File namefilename = f_path/f_namesheetname = 'example' # change to the name of the worksheet# create Excel objectexcel = win32.gencache.EnsureDispatch('Excel.Application')# excel can be visible or notexcel.Visible = True # open Excel Workbook   wb = excel.Workbooks.Open(filename)# set cell horizontal alignmentwb.Worksheets("example").Columns("A:D").HorizontalAlignment = win32c.xlLeft
In the example above, the value assigned is xlLeft, as it is Excel Constant, we will have to add win32c (as I save win32.constants as win32c) in front of the constants. The result of the script above is shown in the figure below.
The horizontal alignment of the header was centre and now aligned to left.
The standard syntax of writing the Excel VBA into Python is to organize the code in this way:
Object.Property = win32c.ExcelConstant
The syntax of pywin32 is like translating Excel VBA into Python syntax. So you can always refer to Excel VBA documentation for the Object, methods and property available in Excel. When the documentation is not enough, you can simply record Macro for that particular process and refer to the step info.
In my work, I use Excel Macro to obtain the color code for the font color (expression.Columns.Font.Color) and cell fill color (expression.Columns.Interior.Color) for a particular column.
In the GIF above, the 255 represent the color code for Red, we can use it in Python script to change the color of a cell.
We can always record Excel Macro when we not sure which Methods or Properties to use during a certain situation.
In this section, I will share the methods and properties that I use frequently. Same with the examples above, we need to import libraries and create excel object before we run any script that executes Excel methods or modifies Excel properties. The Excel File used in the examples below is still the one that we created in the section earlier.
import pandas as pdfrom pathlib import Pathimport win32com.client as win32win32c = win32.constantsf_path = Path.cwd() # change to the path to the Excel filef_name = 'example.xlsx' # Excel File namefilename = f_path/f_namesheetname = 'example' # change to the name of the worksheet# create Excel objectexcel = win32.gencache.EnsureDispatch('Excel.Application')# excel can be visible or notexcel.Visible = True # open Excel Workbook   wb = excel.Workbooks.Open(filename)
Auto Filter
Auto Filter
Add Filter to every column.
wb.Sheets(sheetname).Range("A1").AutoFilter(Field = 1)
2. AutoFit
Fit the column width according to the word length.
wb.Sheets(sheetname).Columns("A:I").AutoFit()
3. Copy
Copy the value of a range of cells to another location.
wb.Sheets(sheetname2).Range("A1:E10").Copy(Destination = wb.Sheets(sheetname).Range("F1"))
4. Copy + Paste Special
Copy format of one sheet to another sheet.
# copy paste FORMAT only, which include font colour, fill colour, and borderwb2.Sheets(sheetname).Range("A:D").Copy()wb.Sheets(sheetname).Range("A1").PasteSpecial(Paste=win32c.xlPasteFormats)
5. Fill Down (Fill Left, Fill Right, Fill Up)
Fill the formula into a range of cell. For example, fill from cell A2 to A1000 as shown in the script below.
wb.Sheets(sheetname).Range("A2:A1000").FillDown()
6. Insert
Insert a new column by shifting the original column down (to a later index, for example, Column C to Column D).
# insert a new row by shifting the original row downwb.Sheets(sheetname).Rows("1:2").Insert(win32c.xlShiftDown)
Column Width
Column Width
Adjust the column width.
wb.Sheets(sheetname).Columns("A").ColumnWidth = 10
2. Font / Interior Color
The font contains more attributes, for example, font color, font size and font name.
There are two methods to assign color, color code and color index. For color code, it’s a long number that you only can obtain by referring to Excel Macro step info. While for Color Index you can create a color table in Excel, and find the index of the color you want.
wb.Worksheets(sheetname).Columns("K:K").Font.Color = -4165632wb.Worksheets("sheetname").Columns("K:K").Interior.Color = 10092492
3. Formula
Assign a formula to the cell.
wb.Sheets(sheetname).Range(f"{column_insert}2").Formula = "=A2+B2"# if you wish to create a functioncolumn1 = "A"column2 = "B"operation = "+" # math operation to perform between 2 columnsdef math_op(column1, column2, operation):  wb.Sheets(sheetname).Range(f"{column_insert}2").Formula =   f"={column1}2{operation}{column2}2"math_op(column1, column2, operation)
The ‘2’ between {column1} and {operation}, and the ‘2’ after {column2} referring to row number 2.
4. Horizontal Alignment
Adjust cell horizontal alignment. There are three types of alignment, which are xlLeft, xlCenter and xlRight.
wb.Worksheets("sheetname").Rows("1:1").HorizontalAlignment = win32c.xlLeft
5. Number Format
Set the format of numerical value. For NumberFormat value, you can obtain it via Excel Macro.
wb.Worksheets("sheetname").Columns("O:O").NumberFormat= '_("$"* #,##0_);_("$"* (#,##0);_("$"* "-"_);_(@_)'
Pywin32 allow us to set up Excel Report like we are doing it manually. We can set formula with reference cells (Example: O2 = P2+Q2), create pivot tables, set cell formats (Example: cell colors, font colors, column width, etc.), set filter and more.
We can automate a report that has to generate periodically with format set up nicely via pywin32. We only have to create all the methods or set up the properties in Python script once.
Moreover, we can enlarge this automation. For example, we may need to send out this Excel report via email after it is done. We can simply add another line(s) of the script to send up the reports.
Lastly, whenever you are in dilemma, remember the standard syntax for Python pywin32 is as follow.
Object.Method(Parameter = parameter)
Object.Property = value
Object.Property = win32c.ExcelConstant
If you are a video person, here’s a video for the key concept you should know to get started to automate your Excel report with PyWin32 libraries.
Subscribe on YouTube
If you are interested in knowing how to automate creating a Pivot Table and extract the filtered data from Pivot Table to pandas DataFrame, Automate Pivot Table with Python (Create, Filter and Extract).
If you are interested to create a chart or PivotChart with Python and export it as an image automatically, Automate Excel Chart with Python.
If your job used Google Sheet instead of Microsoft Excel, you may refer to this article, “Automate Google Sheet Report” for the possible automation to be made.
If you are interested to know how to use pywin32 to access Microsoft Outlook mails and download attachment, you may refer to this article, “Automatically Download Email Attachment with Python”.
[1] H. Zhou, Learn Data Mining Through Excel: A Step-by-Step Approach for Understanding Machine Learning Methods, New York: Apress Media, 2020.
[2] How to Create a Pivot Table in Excel with the Python win32com Module — A complete example of creating a pivot table in Excel with Python.
[3] Excel VBA reference — A complete reference on the objects, methods, properties and events in Microsoft Excel.
Congrats and thanks for reading to the end. Hope you enjoy this article. ☺️