<PAGE>   1
                                                                   EXHIBIT 10.84

                              SEVENTH AMENDMENT TO
                  FIRST AMENDED AND RESTATED CREDIT AGREEMENT


         THIS SEVENTH AMENDMENT TO FIRST AMENDED AND RESTATED CREDIT AGREEMENT
(the "Amendment"), dated as of May 5, 1999, is entered into between the Lenders
party hereto, BANKERS TRUST COMPANY, a New York banking corporation, as agent
for the Lenders (the "Agent"), DORAL FINANCIAL CORPORATION (formerly known as
First Financial Caribbean Corporation), a corporation organized under the laws
of the Commonwealth of Puerto Rico ("DFC"), and DORAL MORTGAGE CORPORATION, a
corporation organized under the laws of the Commonwealth of Puerto Rico and a
wholly-owned subsidiary of DFC ("DMC", and together with DFC, each a "Borrower"
and collectively, the "Borrowers"), with reference to the First Amended and
Restated Credit Agreement, dated as of September 25, 1996, between the
Borrowers, the Agent and the lenders party thereto (as amended by the First
Amendment to First Amended and Restated Credit Agreement and Certain Other Loan
Documents dated as of January 8, 1997, the Second Amendment to First Amended
and Restated Credit Agreement dated as of March 28, 1997, the Third Amendment
to First Amended and Restated Credit Agreement dated as of June 27, 1997, the
Fourth Amendment to First Amended and Restated Credit Agreement dated as of
November 5, 1997, the Fifth Amendment to First Amended and Restated Credit
Agreement dated as of June 26, 1998, and the Sixth Amendment to First Amended
and Restated Credit Agreement dated as of November 23, 1998, and as further
amended, supplemented or otherwise modified from time to time, the "Credit
Agreement"). All capitalized terms used but not otherwise defined herein shall
have the meanings given such terms in the Credit Agreement.

         The Lenders, the Agent and the Borrowers wish to amend the Credit
Agreement as set forth herein.

         ACCORDINGLY, the parties hereto agree as follows:

         Section 1.        Addition of New Lenders and Amendment of Commitments.
As of the effective date of this Amendment as determined in accordance with
Section 5 below, each of Credit Lyonnais, New York Branch and Hibernia National
Bank shall (i) become a party to the Credit Agreement and a Facility 1 Lender
thereunder, (ii) be entitled to all rights, benefits and privileges accorded a
Facility 1 Lender under the Credit Agreement and under the other Loan
Documents, (iii) be subject to all obligations of a Facility 1 Lender under the
Credit Agreement and under the other Loan Documents, and (iv) have the
respective Facility 1 Commitments set forth opposite its name on the signature
pages hereto, as such Commitments may be reduced from time to time pursuant to
Section 8.6(c) of the Credit Agreement. The respective addresses for notices
for Credit Lyonnais, New York Branch and Hibernia National Bank are set forth
on Schedule X hereto. The Commitments of each of the existing Lenders shall be
as set forth opposite such Lender's name on the signature pages hereto, as such
Commitments may be reduced from time to time pursuant to Section 8.6(c) of the
Credit Agreement.


<PAGE>   2



         Section 2.        Amendment to Section 2.1 of the Credit Agreement. 
Section 2.1(a)(i) of the Credit Agreement shall be amended by replacing the
phrase "fifty percent (50%) of the aggregate Facility 1 Commitments then in
effect" with "$62,500,000".

         Section 3.        Other Amendments to Credit Agreement. Section 8.6 of
the Credit Agreement shall be amended by adding thereto the following new
subsections (g) and (h) immediately following subsection (f) thereof:

         "(g)    So long as no Potential Default or Event of Default has 
occurred and is continuing, the Borrowers may at any time propose that one or
more commercial banks (each, an "Applicant Lender") become an additional Lender
hereunder. At such time, the Borrowers shall notify each of the Lenders of (i)
the identity of such Applicant Lender, (ii) the proposed increase in the
aggregate Commitments resulting from the addition of such Applicant Lender as a
Lender hereunder, and (iii) such Applicant Lender's proposed Commitments (which
must be not less than $15,000,000 in the case of a Facility 1 Commitment and
not less than $1,000,000 in the case of a Facility 2 Commitment). Upon the
Applicant Lender's addition as a Lender hereunder, such Applicant Lender shall
become a party to this Agreement and a Lender hereunder, shall be entitled to
all rights, benefits and privileges accorded a Lender under this Agreement and
under the other Loan Documents, and shall be subject to all obligations of a
Lender under this Agreement and under the other Loan Documents. Notwithstanding
the foregoing or anything else contained herein, the aggregate amount of the
Facility 1 Commitments of all Lenders shall not exceed $250,000,000 at any time
and the aggregate amount of the Facility 2 Commitments of all Lenders shall not
exceed $15,000,000 at any time.

         (h)      The addition of any Applicant Lender as a Lender hereunder 
shall become effective upon the occurrence of each of the following events:

         (i)      the Borrowers and the Agent shall have given their prior
                  written consent to such Applicant Lender, which consent shall
                  not be unreasonably withheld, delayed or conditioned;

         (ii)     such Applicant Lender, the Borrowers and the Agent shall have
                  mutually agreed on the date (the "Adjustment Date") on which
                  such Applicant Lender shall become a Lender hereunder and
                  under the other Loan Documents; and

         (iii)    on such Adjustment Date:

                  (A)      such Applicant Lender, the Borrowers and the Agent
                           shall execute and deliver to each of the other
                           signatories thereto an Additional Lender Agreement
                           in substantially the form of Exhibit W hereto and
                           the Agent shall deliver a copy of such executed
                           Additional Lender Agreement and a copy of the
                           Commitment Notice attached as Schedule I thereto to
                           each of the Lenders;


                                      -2-
<PAGE>   3




                  (B)      the Borrowers shall execute and deliver to such
                           Applicant Lender (x) a duly executed Facility 1 Note
                           and/or a duly executed Facility 2 Note, as the case
                           may be, (y) a reliance letter from Puerto Rico
                           counsel to the Borrowers addressed to such Applicant
                           Lender and in form and substance satisfactory to
                           such Applicant Lender, providing that such Applicant
                           Lender may rely on the opinion letters of such
                           counsel addressed to the existing Lenders and dated
                           October 10, 1996 and May 7, 1997, as if such opinion
                           letters were addressed directly to such Applicant
                           Lender, and (z) certified copies of the resolutions
                           of the Board of Directors of each of the Borrowers
                           evidencing the authorization of such Borrower to
                           enter into such Additional Lender Agreement and
                           consummate the transactions and matters contemplated
                           thereby; and

                  (C)      no Potential Default or Event of Default shall have
                           occurred and be continuing as of such date."

         Section 4.        Addition of New Exhibit W. Exhibit W in the form of 
Exhibit W hereto shall be added to the Credit Agreement immediately following
Exhibit V thereto.

         Section 5.        Representations and Warranties. The Borrowers 
represent and warrant that, on and as of the date hereof, all of the
representations and warranties made by them in the Credit Agreement and the
other Loan Documents are true and correct as if made on and as of the date
hereof and no Potential Default or Event of Default has occurred and is
continuing.

         Section 6.        Effectiveness. This Amendment shall become effective
as of the date hereof upon delivery to the Agent of (i) counterparts of this
Amendment, duly executed and delivered by the parties hereto, (ii) a duly
executed Facility 1 Note for each of Credit Lyonnais, New York Branch and
Hibernia National Bank, (iii) a reliance letter from Puerto Rico counsel to the
Borrowers addressed to Credit Lyonnais, New York Branch and Hibernia National
Bank and in form and substance satisfactory to Credit Lyonnais, New York Branch
and Hibernia National Bank providing that Credit Lyonnais, New York Branch and
Hibernia National Bank may rely on the opinion letters of such counsel
addressed to the existing Lenders and dated October 10, 1996 and May 7, 1997,
as if such opinion letters were addressed directly to Credit Lyonnais, New York
Branch and Hibernia National Bank, respectively, and (iv) certified copies of
the resolutions of the Board of Directors of each of the Borrowers evidencing
the authorization of such Borrower to enter into this Amendment and consummate
the transactions and matters contemplated hereby.

         Section 7.        Counterparts. This Amendment may be executed in any
number of counterparts, all of which taken together shall constitute one
agreement, and any party hereto may execute this Amendment by signing any such
counterpart.


                                      -3-
<PAGE>   4



         Section 8.        GOVERNING LAW. THIS AMENDMENT SHALL BE GOVERNED BY 
AND CONSTRUED IN ACCORDANCE WITH THE LAWS OF THE STATE OF NEW YORK.

         Section 9.        Miscellaneous. Except as expressly amended hereby, 
the Credit Agreement and the other Loan Documents shall remain in full force
and effect. Nothing contained herein shall operate as a waiver of any right,
power or remedy of the Agent or the Lenders under the Credit Agreement or any
other Loan Document, nor constitute a waiver of any provision of the Credit
Agreement or any other Loan Document.


         IN WITNESS WHEREOF, the parties hereto have caused this Amendment to
be executed as of the day and year first above written.


                                DORAL FINANCIAL CORPORATION,
                                as a Borrower

                                By:      /s/ Mario S. Levis      
                                         --------------------------------------
                                Name:    Mario S. Levis
                                Title:   Executive Vice President and Treasurer


                                DORAL MORTGAGE CORPORATION,
                                as a Borrower

                                By:      /s/ Mario S. Levis
                                         --------------------------------------
                                Name:    Mario S. Levis
                                Title:   Executive Vice President and Treasurer


Facility 1 Commitment:          BANKERS TRUST COMPANY,
$39,000,000                     as Agent and as a Lender
Facility 2 Commitment:
$2,000,000
                                By:      /s/ Kevin M. McCann
                                         --------------------------------------
                                Name:    Kevin M. McCann
                                Title:   Managing Director


                                      -4-
<PAGE>   5




Facility 1 Commitment:          FIRST UNION NATIONAL BANK,
$39,000,000                     as a Lender
Facility 2 Commitment:
$2,000,000
                                By:      /s/ R. STEVEN HALL
                                         --------------------------------------
                                Name:    R. Steven Hall
                                         --------------------------------------
                                Title:   Vice President
                                         --------------------------------------


Facility 1 Commitment:          BANKBOSTON, N.A. (formerly known as
$29,000,000                     The Bank of Boston), as a Lender
Facility 2 Commitment:
$2,000,000
                                By:      /s/ PAUL A. CHMIELINSKI
                                         --------------------------------------
                                Name:    Paul A. Chmielinski
                                         --------------------------------------
                                Title:   Vice President
                                         --------------------------------------



Facility 1 Commitment:          THE BANK OF NEW YORK,
$25,000,000                     as a Lender
Facility 2 Commitment:
$2,000,000
                                By:      /s/ ROBERT A. TWEED
                                         --------------------------------------
                                Name:    Robert A. Tweed
                                         --------------------------------------
                                Title:   Vice President
                                         --------------------------------------



Facility 1 Commitment:          NATIONAL CITY BANK OF KENTUCKY,
$25,000,000                     as a Lender
Facility 2 Commitment:
$2,000,000
                                By:      /s/ ROBERT J. OGBURN
                                         --------------------------------------
                                Name:    Robert J. Ogburn
                                         --------------------------------------
                                Title:   Vice President
                                         --------------------------------------


Facility 1 Commitment:          CREDIT LYONNAIS, NEW YORK BRANCH
$23,437,500                     as a Lender

                                By:      /s/ SEBASTIAN ROCCO
                                         --------------------------------------
                                Name:    Sebastian Rocco
                                         --------------------------------------
                                Title:   Senior Vice President
                                         --------------------------------------


                                      -5-
<PAGE>   6




Facility 1 Commitment:          HIBERNIA NATIONAL BANK,
$19,500,000                     as a Lender

                                By:      /s/ EDWARD K. SANTOS
                                         --------------------------------------
                                Name:    Edward K. Santos
                                         --------------------------------------
                                Title:   Vice President
                                         --------------------------------------




                                      -6-
<PAGE>   7

                                                                     SCHEDULE X


                              ADDRESS FOR NOTICES



Credit Lyonnais, New York Branch
1301 Avenue of the Americas
New York, NY 10019
Attn:  Paul Connolly
Telephone:  (212) 261-3885
Fax:  (212) 261-3438


Hibernia National Bank
P.O. Box 61540
New Orleans, LA 70161
Attn:  Stephanie F. Tyner
Telephone:  (504) 533-3345
Fax:  (504) 533-6242

<PAGE>   8

                                                                      EXHIBIT W

                          ADDITIONAL LENDER AGREEMENT


                  THIS ADDITIONAL LENDER AGREEMENT (the "Agreement") is dated
as of _________________, ____, between _____________________ (the "Applicant 
Lender"), BANKERS TRUST COMPANY, a New York banking corporation, as agent for
the Lenders (the "Agent"), DORAL FINANCIAL CORPORATION (formerly known as First
Financial Caribbean Corporation), a corporation organized under the laws of the
Commonwealth of Puerto Rico ("DFC"), and DORAL MORTGAGE CORPORATION, a
corporation organized under the laws of the Commonwealth of Puerto Rico and a
wholly-owned subsidiary of DFC, with reference to the First Amended and
Restated Credit Agreement, dated as of September 25, 1996, between the
Borrowers, the Agent and the lenders party thereto (as amended, supplemented or
otherwise modified from time to time, the "Credit Agreement"). All capitalized
terms used but not otherwise defined herein shall have the meanings given such
terms in the Credit Agreement.

                  A.       The Applicant Lender desires to become a "Lender" 
under the Credit Agreement.

                  B.       Pursuant to Section 8.6 of the Credit Agreement, the
Applicant Lender has been approved by the Borrowers and the Agent for inclusion
as a Lender under the Credit Agreement and the other Loan Documents.

                  ACCORDINGLY, the parties hereto agree as follows:

                  1.       As of the Adjustment Date, the Applicant Lender shall
become a party to the Credit Agreement and shall be a "Lender" thereunder and
under the other Loan Documents, shall be entitled to all rights, benefits and
privileges accorded a Lender under the Credit Agreement and under the other
Loan Documents, and shall be subject to all obligations of a Lender under the
Credit Agreement and under the other Loan Documents. The Commitment Notice
attached as Schedule I hereto sets forth the Commitments of each of the Lenders
after giving effect to the inclusion of the Applicant Lender as a Lender under
the Credit Agreement and the other Loan Documents.

                  2.       The address for notices of the Applicant Lender shall
initially be as set forth beneath its signature below.

                  3.       This Agreement and the rights and obligations of the
parties hereunder shall be governed by and construed in accordance with the
laws of the State of New York.

                  4.       This Agreement may be executed in any number of
counterparts, each of 


<PAGE>   9

which when so executed and delivered shall be an original, but all of which
together shall constitute one and the same instrument.

                  5.       This Agreement, when executed by each of the parties
hereto, shall constitute an amendment of the Credit Agreement and the other
Loan Documents consistent with the Commitment Notice referred to in Section 1
above.


                  EXECUTED as of the day and year first above written.



                                DORAL FINANCIAL CORPORATION,
                                as a Borrower

                                By:   
                                         --------------------------------------
                                Name:    
                                         --------------------------------------
                                Title:   
                                         --------------------------------------


                                DORAL MORTGAGE CORPORATION,
                                as a Borrower

                                By:   
                                         --------------------------------------
                                Name:    
                                         --------------------------------------
                                Title:   
                                         --------------------------------------


                                BANKERS TRUST COMPANY,
                                as Agent and as a Lender

                                By:   
                                         --------------------------------------
                                Name:    
                                         --------------------------------------
                                Title:   
                                         --------------------------------------


                                [----------------------------------],
                                as Applicant Lender

                                By:   
                                         --------------------------------------
                                Name:    
                                         --------------------------------------
                                Title:   
                                         --------------------------------------

                                Address:


<PAGE>   10

                                                                  Schedule I to
                                                    Additional Lender Agreement


                               COMMITMENT NOTICE


                  This Commitment Notice, dated ____________, _____, is
delivered in connection with that certain First Amended and Restated Credit
Agreement, dated as of September 25, 1996, between BANKERS TRUST COMPANY, a New
York banking corporation, as agent for the Lenders (the "Agent"), DORAL
FINANCIAL CORPORATION (formerly known as First Financial Caribbean
Corporation), a corporation organized under the laws of the Commonwealth of
Puerto Rico ("DFC"), and DORAL MORTGAGE CORPORATION, a corporation organized
under the laws of the Commonwealth of Puerto Rico and a wholly-owned subsidiary
of DFC, and the Lenders party thereto from time to time (as amended,
supplemented or extended from time to time, the "Credit Agreement").
Capitalized terms not otherwise defined herein shall have the meanings given to
such terms in the Credit Agreement.

                  The Agent hereby notifies each Lender as follows:

         1.       __________________  has been added as a Lender under the 
                  Credit Agreement effective as of _____________, ____ (the
                  "Adjustment Date").

         2.       As of the Adjustment Date, the respective Commitments of each
                  Lender, after giving effect to the addition of _______________
                  as a Lender, are as follows:

         [INSERT COMMITMENTS OF EACH LENDER]



                                BANKERS TRUST COMPANY,
                                as Agent

                                By:   
                                         --------------------------------------
                                Name:    
                                         --------------------------------------
                                Title:   
                                         --------------------------------------