 Construction on the Brookville Tunnel, or the "middle" or "second tunnel" as it was then called, was begun sometime between 1849 and 1851. The earth encountered at Brookville was of a much weaker quality than at the Greenwood Tunnel, rendering the digging of the tunnel both an extremely dangerous and difficult task. Although the eastern section of the ridge was composed of firm and sturdy rock, the majority of the tunnel passed through a mixture of frail slate that was formed at a steep angle, soapstone, and clay. When exposed to air, this mixture would often disintegrate causing large amounts of earth to fall into the tunnel bore. On two occasions during the tunnel's construction, though great care was taken to provide temporary support, major cave-ins occurred forming craters on top of the ridge, as much as 150 feet (46 m) above the tunnel. Another slide occurred on the outside of the western portal that blocked the entrance to the tunnel and prevented workers from repairing the damage from the two internal cave-ins. Rocks often fell down upon the timbers under which the men worked, creating sounds described as "resembling that of distant thunder". The weakness of the earth, especially in the western sections, necessitated the implementation of a strong arch throughout approximately 500 feet (150 m) of the tunnel and a durable brick lining. Crozet described the nature of the work on the tunnel in an 1854 letter:
