The process of dividing a huge dataset (DataFrame) into multiple small datasets while writing to disk (based on columns) is called PySpark Partition. On a filesystem, data partitioning can help to improve the performance of queries while dealing with large datasets in the Data lake. This is because transformations on partitioned data run smoothly and quickly, so the speed of query execution is improved. There are two partitioning methods supported by PySpark:  Partition in memory: Using this method, partition or repartition the DataFrame by invoking coalesce() or repartition() transformations.  Partition on disk: You can partition data while writing the DataFrame to disk based on columns using PartitionBy().