Florida Avenue was the original boundary of the Federal City of Washington which was designed by L'Enfant. Until the early 1900's there was a distinction between the City of Washington and the District of Columbia. The Federal City was generally the area south of Florida Avenue. You will notice on a map that the streets of Columbia Heights don't conform to L'Enfants grid plan. Columbia Heights grew as a rural area and then suburb outside Washington.
Today there is no distinction between Washington and the District of Columbia.
Legally the Federal City has been reduced to an enclave around Federal buildings downtown. The western boundary of Columbia Heights is 16th Street. At the edge where 16th Street meets Florida Avenue is Meridian Hill Park . In my opinion it is the most beautiful planned Park in the City. It is a French-Italian formal garden with statuary, fountains and pools.