<PAGE>

                                                                    Exhibit 10.5


                       FOURTH NOTE MODIFICATION AGREEMENT

               THIS FOURTH NOTE MODIFICATION AGREEMENT (this "MODIFICATION") is
made as of this 5th day of November, 1999 by and between LASALLE BANK NATIONAL
ASSOCIATION (formerly known as LaSalle National Bank) (the "LENDER") and PRIME
GROUP REALTY, L.P., a Delaware limited partnership (the "BORROWER").

                                    RECITALS:

        1. The Lender, the Borrower, and Prime Group Realty Trust (the
"COMPANY") have previously executed and delivered a loan agreement dated January
28, 1998, as amended by that certain Amended and Restated Loan Agreement dated
as of October 1, 1998, as further amended by that certain Second Amended and
Restated Loan Agreement dated as of March 23, 1999, and as further amended by an
Amendment to Second Amended and Restated Loan Agreement dated as of June 30,
1999 (the "Amended Loan Agreement"), setting forth the terms and conditions of a
revolving line of credit in favor of Borrower (the "LOAN").

        2. In connection with the Second Amended and Restated Loan Agreement,
Borrower has previously executed and delivered to Lender a certain Revolving
Loan Note dated January 28, 1998, as amended by a Note Modification Agreement
dated as of October 1, 1998, and as further amended by a Second Note
Modification Agreement dated as of March 23, 1999, and as further amended by a
Third Note Modification Agreement dated as of June 30, 1999 (the "MODIFIED
NOTE"), in the original principal face amount of $15,000,000, payable to the
order of Lender, and evidencing the Loan.

        3. 475 Superior Avenue, L.L.C. ("MORTGAGOR") is a guarantor of the Loan,
pursuant to the Guaranty (defined below).

        4. To secure Mortgagor's obligations as a Guarantor under that certain
Guaranty of Payment and Performance dated January 28, 1998, as modified pursuant
to that certain First Amendment to Guaranty dated February 17, 1998, as further
modified by that certain Release and Reaffirmation of Guaranty dated as of
October 1, 1998, as further modified by that certain Second Reaffirmation of
Guaranty and Addition of Guarantor Subsidiaries dated as of March 23, 1999, as
further modified by that certain Third Reaffirmation of Guaranty dated as of
June 30, 1999, and as further modified by that certain Fourth Reaffirmation of
Guaranty of even date herewith (the "GUARANTY"), and in consideration of the
Loan made to Borrower, Mortgagor has previously executed and delivered to Lender
the following documents, each dated January 28, 1998, as amended by that certain
First Amendment to Loan Documents dated as of October 1, 1998 and recorded in
the real property records of Lake County, Indiana as Document Number 98093313,
as further amended by that certain Second Amendment to Loan Documents dated as
of March 23, 1999 and recorded in the real property records of Lake County,
Indiana as Document Number 99029971, and as further amended by that certain
Third Amendment to Loan Documents dated as of June 30, 1999 and recorded in the
real property records of Lake County, Indiana as Document Number 99057426:

               (a) that certain Mortgage, Assignment of Leases and Rents,
               Security Agreement, and Financing Statement, executed and
               delivered by the Mortgagor,


<PAGE>



               recorded in the real property records of Lake County, Indiana as
               Document Number 98006633 and encumbering certain real property
               described therein, and as described on Exhibit A hereto (the
               "MORTGAGE");

               (b) that certain Assignment of Leases and Rents, executed and
               delivered by the Mortgagor, recorded in the real property records
               of Lake County, Indiana as Document Number 98006634 and
               encumbering certain real property described therein, and as
               described in Exhibit A hereto (the "ASSIGNMENT");

               (c) that certain Security Agreement executed and delivered by
               Mortgagor encumbering certain collateral described therein (the
               "SECURITY AGREEMENT");

        5. The Lender, the Borrower and the Company have amended the Amended
Loan Agreement pursuant to that Second Amendment to Second Amended and Restated
Loan Agreement dated of even date herewith (the "SECOND AMENDMENT TO SECOND
AMENDED AND RESTATED LOAN AGREEMENT"; the Amended Loan Agreement, as amended by
the Second Amendment to Second Amended and Restated Loan Agreement, is
hereinafter referred to as the "Second Amended Loan Agreement").

        6. The Lender and Mortgagor have further amended the Mortgage,
Assignment, and Security Agreement pursuant to that certain Fourth Amendment to
Loan Documents dated of even date herewith (the "FOURTH AMENDMENT TO LOAN
DOCUMENTS").

               NOW THEREFORE, in consideration of the foregoing recitals, and
for other good and valuable consideration, the receipt and sufficiency of which
are hereby acknowledged, the Lender and Borrower agree as follows:

        A.     AMENDMENT OF THE MODIFIED NOTE

        The Modified Note is hereby amended as follows:

               (1) Any and all references in the Modified Note to the Amended
               Loan Agreement shall be deemed to refer to the "Second Amended
               Loan Agreement, as may be further amended from time to time."

               (2) Any and all references to the Mortgage, Assignment, or
               Security Agreement shall be deemed to refer to the Mortgage,
               Assignment, or Security Agreement "as amended by the Fourth
               Amendment to Loan Documents and as may be further amended from
               time to time."

        B. FULL FORCE AND EFFECT. All of the provisions, rights, powers, and
remedies contained in the Note shall stand and shall remain unchanged and in
full force and effect, except to the extent specifically amended hereby.


                                       -2-


<PAGE>



         C. COUNTERPARTS. This Modification may be executed in any number of
counterparts, each of which shall be deemed an original, but which together
shall constitute one and the same instrument.

            [the remainder of this page is intentionally left blank]


                                       -3-


<PAGE>


               IN WITNESS WHEREOF, this Modification has been duly executed
effective as of the date first written above.

                                    LENDER:

                                    LASALLE BANK NATIONAL ASSOCIATION, a
                                    national banking association

                                    By:     _________________________________
                                    Name:   _________________________________
                                    Its:    _________________________________

                                    BORROWER:

                                    PRIME GROUP REALTY, L.P., a Delaware
                                    limited partnership

                                    By:     PRIME GROUP REALTY TRUST, a
                                            Maryland trust, its general partner

                                            By:   /s/ Patrick L. McGaughy
                                                  ----------------------------
                                            Name: Patrick L. McGaughy
                                            Its:  Vice President


                                       -4-