Getting all CSV files from a directory using Python - GeeksforGeeks

16 May, 2021
Python provides many inbuilt packages and modules to work with CSV files in the workspace. The CSV files can be accessed within a system’s directories and subdirectories and modified or edited. The CSV file contents can both be printed on the shell, or it can be saved in the form of the dataframe and played with later.
In this article, we will see how to iterate through n number of CSV files contained within a directory (folder/ path) consisting of different types of files, and to work with the contents of these files. So, we apply two different methods to our task. 
Input Directory for both the approaches :
CSV Used:
CSV 1:
CSV 2:
CSV 3:
Initially, the path of the source directory is specified, in this case, the folder “csvfoldergfg” using path variable.
path = "csvfoldergfg"
In order to locate all CSV files, whose names may be unknown, the glob module is invoked and its glob method is called. It is supplied with the path using glob.glob(path). This returns all the CSV files’ list located within the path. The regex used is equivalent to *.csv, which matches all files for an extension .csv.
glob.glob(path, '*.csv')
An iteration is then performed over these files using the for loop and the content is read into a data frame, using the read_csv() method of the pandas library. The information fetched using this method can then be manipulated.
read_csv(file_contents)
The following code was executed on the local machine, where both the script and the directory whose path is specified are stored in the same working directory:
Python3
# importing the required modulesimport globimport pandas as pd  # specifying the path to csv filespath = "csvfoldergfg"  # csv files in the pathfiles = glob.glob(path + "/*.csv")  # defining an empty list to store # contentdata_frame = pd.DataFrame()content = []  # checking all the csv files in the # specified pathfor filename in files:        # reading content of csv file    # content.append(filename)    df = pd.read_csv(filename, index_col=None)    content.append(df)  # converting content to data framedata_frame = pd.concat(content)print(data_frame)
Output:
Initially the path of the source directory is specified, in this case, the folder “csvfoldergfg” using dir_name string variable.
dir_name = "csvfoldergfg"
In order to locate all the files, whose names may be unknown, the os module is invoked, and its listdir() method is called. It is supplied with the path using os.listdir(path). This returns all the files’ list located within the path.
os.listdir(dir_name)
An iteration is then performed over these files using the for loop and the content is read into a data frame, using the read_csv() method of the pandas library. The information fetched using this method can then be manipulated.
read_csv(file_contents)
The following code is executed on the local machine :
Python3
# importing the required packaged # in pythonimport pandas as pdimport osdir_name = "csvfoldergfg"  # specifying an empty list for contentcontent = []for file in os.listdir(dir_name):          # reading content into data frame    df = pd.read_csv(file)    df_list.append(df)  final_content = df.append(df for df in df_list)print(final_content)
Output:
Picked
python-csv
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Python Dictionary
Read a file line by line in Python
How to Install PIP on Windows ?
Enumerate() in Python
Different ways to create Pandas Dataframe
Iterate over a list in Python
*args and **kwargs in Python
Reading and Writing to text files in Python
Convert integer to string in Python
Check if element exists in list in Python