What is the best way to combine rows in a large dataset in excel a report I pull gives me an excel spreadsheet that splits the data for each entry across three rows in excel. I'm trying to figure out the best way to combine the three rows into one row so each field is in it's own column.
Each three row cluster is separated by a blank row and each of the data rows has five columns. The first cluster starts on row 4.
I have a macro (shown below) that does this correctly, but not efficiently. The spreadsheets I get have many (up to a million) rows in them.
I was originally using the cut and paste commands and that was really slow. I found that directly setting .value make it quite a bit faster but this is still way to slow.
I think that the right answer is to do all of the manipulation in memory and write to the actual excel range only once, but I'm at the limits of my VBA foo.
Option Explicit
Sub CombineRows()
    Application.ScreenUpdating = False

    Dim currentRow As Long
    Dim lastRow As Long
    Dim pasteColumn As Long
    Dim dataRange As Range
    Dim rowEmpty As Boolean
    Dim firstOfGroup As Boolean
    Dim data As Variant
    Dim rw As Range

    pasteColumn = 6
    rowEmpty = True
    firstOfGroup = True
    currentRow = 4
    lastRow = 30
    Set dataRange = Range(Cells(currentRow, 1), Cells(lastRow, 5))

    For Each rw In dataRange.Rows
Debug.Print rw.Row
If WorksheetFunction.CountA(Range(Cells(rw.Row, 1), Cells(rw.Row, 5))) = 0 Then
If rowEmpty Then Exit For
currentRow = rw.Row + 1
rowEmpty = True
Else
If Not rowEmpty Then
Range(Cells(currentRow, pasteColumn), Cells(currentRow, pasteColumn + 4)).value = Range(Cells(rw.Row, 1), Cells(rw.Row, 5)).value
Range(Cells(rw.Row, 1), Cells(rw.Row, 5)).value = ""
Debug.Print "pasteColumn:"; pasteColumn
If pasteColumn = 6 Then
pasteColumn = 11
ElseIf pasteColumn = 11 Then
pasteColumn = 6
End If
End If
rowEmpty = False
End If
    Next
    Application.ScreenUpdating = True
End Sub

Update: After I posted this, I noticed that I still had those Debug.Print statements in there. Once I removed those, the performance improved from execution times on the order of hours to a minute or two.
I still thing that this is unnecessarily slow so I'm still interested in any answer that can explain the right way to minimize the VBA <-> excel interactions.

If I understand correctly your question, you want to copy some data.
I recommend you to use an array.
Sub data()
    Dim data() As String 'Create array  
    Dim column as integer
    column = 0
    For i = 0 To 100000 'See how many columns are in the line
If IsEmpty(Cells(rowNum, i+1)) = False Then
column = column + 1
Else
Exit For
End If
    Next
    ReDim date(column) As String 'Recreat the array, with the excat column numer
    For i = 0 To column - 1
data(i, j) = Cells(rowNum, i + 1) 'Puts data into the array
    Next
End sub()

And now you just have to insert the data from the array to the correct cell.

@Cubbi is correct. You can use an array to do all of your data manipulation and then write to the worksheet only once at the end. I've adapted your code to use an array to combine the three rows into a single row for each of the groups. Then at the end it selects "Sheet2" and pastes in the collected data. Note, this is not an in-place solution like yours, but it is super fast:
Option Explicit
Sub AutitTrailFormat()
    Application.ScreenUpdating = False

    Dim dataArray() As String
    Dim currentRow As Long
    Dim lastRow As Long
    Dim pasteColumn As Long
    Dim dataRange As Range
    Dim rowEmpty As Boolean
    Dim firstOfGroup As Boolean
    Dim data As Variant
    Dim rw As Range
    Dim i, j, k As Long
    Dim Destination As Range

    pasteColumn = 6
    rowEmpty = True
    firstOfGroup = True
    currentRow = 4
    lastRow = ActiveSheet.Cells(Rows.Count, 1).End(xlUp).Row
    Worksheets("Sheet1").Select
    Set dataRange = Worksheets("Sheet1").Range(Cells(currentRow, 1), Cells(lastRow, 5))
    data = dataRange.Value
    ReDim dataArray(UBound(data, 1), 15)

    j = 1
    k = 1
    For i = 1 To UBound(data, 1)
If data(i, 1) = "" And data(i, 2) = "" And data(i, 3) = "" And data(i, 4) = "" And data(i, 5) = "" Then
j = j + 1
k = 1
Else
dataArray(j, k + 0) = data(i, 1)
dataArray(j, k + 1) = data(i, 2)
dataArray(j, k + 2) = data(i, 3)
dataArray(j, k + 3) = data(i, 4)
dataArray(j, k + 4) = data(i, 5)
k = k + 5
End If
    Next

    Worksheets("Sheet2").Select
    Set Destination = Worksheets("Sheet2").Range(Cells(1, 1), Cells(UBound(dataArray, 1), 16))
    Destination.Value = dataArray
    Application.ScreenUpdating = True
End Sub