<PAGE>   1
                                                                   EXHIBIT 10.14

                          REGISTRATION RIGHTS AGREEMENT

                             Dated January 11, 2000

                                      among

                              CARVER BANCORP, INC.

                        MORGAN STANLEY & CO. INCORPORATED

                                       and

                        PROVENDER OPPORTUNITIES FUND L.P.


<PAGE>   2

                                TABLE OF CONTENTS
                                -----------------
<TABLE>
<CAPTION>
                                                                             PAGE
                                                                             ----

<S>                                                                        <C>
ARTICLE 1

      DEFINITIONS

SECTION 1.01.  Definitions.....................................................1

ARTICLE 2

      REGISTRATION RIGHTS

SECTION 2.01.  Demand Registration Rights......................................3
SECTION 2.02.  Piggy-Back Registration Rights..................................4
SECTION 2.03.  Registration Procedures.........................................5
SECTION 2.04.  Participation in Underwritten Registrations.....................9
SECTION 2.05.  Holdback Agreements.............................................9
SECTION 2.06.  Indemnification.................................................9

ARTICLE 3

      MISCELLANEOUS

SECTION 3.01.  Notices........................................................13
SECTION 3.02.  Amendments; Waivers............................................13
SECTION 3.03.  Successors; Assigns............................................13
SECTION 3.04.  Entire Agreement...............................................13
SECTION 3.05.  Applicable Law.................................................14
SECTION 3.06.  Remedies.......................................................14
SECTION 3.07.  Severability...................................................14
SECTION 3.08.  Fees and Expenses..............................................14
SECTION 3.09.  Counterparts...................................................14
</TABLE>


<PAGE>   3



                          REGISTRATION RIGHTS AGREEMENT

         REGISTRATION RIGHTS AGREEMENT dated January 11, 2000 among Carver
Bancorp, Inc., a Delaware corporation (the "ISSUER"), Morgan Stanley & Co.
Incorporated, a Delaware corporation ("MS"), and Provender Opportunities Fund
L.P., a Delaware limited partnership ("PROVENDER"). Each of MS and Provender is
sometimes hereinafter referred to as a "HOLDER".

         WHEREAS, the Issuer, MS and Provender are parties to a Securities
Purchase Agreement dated January 11, 2000 (the "PURCHASE AGREEMENT") pursuant to
which MS purchased 40,000 shares of Series A Preferred Stock from the Issuer and
Provender purchased 60,000 shares of Series B Preferred Stock from the Issuer;
and

         WHEREAS, the parties hereto desire to provide for certain rights and
obligations relating to the capital stock of the Issuer following the date
hereof.

         NOW, THEREFORE, the parties hereto agree as follows:

                                    ARTICLE 1

                                   DEFINITIONS

         SECTION 1.01. Definitions. (a) The following terms, as used herein,
have the following meanings:

         "BUSINESS DAY" means any day except a Saturday, Sunday or other day on
which commercial banks in the City of New York are authorized by law to close.

         "COMMISSION" means the Securities and Exchange Commission or any
successor commission or agency having similar powers.

         "COMMON SHARES" means shares of the common stock of the Issuer, par
value $0.01 per share.

         "EXCHANGE ACT" means the Securities Exchange Act of 1934, as amended,
and the rules and regulations promulgated thereunder.

         "PERSON" means an individual, partnership, corporation, limited
liability company, trust, joint stock company, association, joint venture, or
any other entity or organization.


<PAGE>   4



         "PUBLIC OFFERING" means any underwritten public offering of equity
securities of the Issuer pursuant to an effective registration statement under
the Securities Act other than pursuant to a registration statement on Form S-4
or Form S-8 or any successor or similar form.

         "REGISTRABLE COMMON SHARES" means Registrable Securities that are
Common Shares.

         "REGISTRABLE SECURITIES" means (i) all Series A Preferred Shares and
all Series B Preferred Shares owned by the Holders and (ii) all Common Shares
owned by the Holders or into which the Series A Preferred Shares or the Series B
Preferred Shares owned by the Holders may be converted. Registrable Securities
shall cease to be Registrable Securities when (i) a registration statement with
respect to the sale of such Registrable Securities shall have become effective
under the Securities Act and such Registrable Securities shall have been
disposed of pursuant to such registration statement, or (ii) such Registrable
Securities shall have ceased to be outstanding.

         "REGISTRATION EXPENSES" means all (i) registration, qualification and
filing fees, (ii) fees and expenses of compliance with securities or blue sky
laws (including reasonable fees and disbursements of a qualified independent
underwriter, if any, counsel in connection therewith and the reasonable fees and
disbursements of counsel in connection with blue sky qualifications of the
Registrable Securities), (iii) printing expenses, (iv) internal expenses of the
Issuer (including, without limitation, all salaries and expenses of officers and
employees performing legal or accounting duties), (v) fees and disbursements of
counsel for the Issuer, (vi) customary fees and expenses for independent
certified public accountants retained by the Issuer (including the expenses of
any comfort letters or costs associated with the delivery by independent
certified public accountants of a comfort letter or comfort letters), (vii) fees
and expenses of any special experts retained by the Issuer in connection with
such registration, (viii) reasonable fees and expenses of one separate firm of
attorneys for the Holders selling securities pursuant to such registration and
(ix) fees and expenses of listing the Registrable Securities on a securities
exchange; but shall not include any underwriting fees or discounts or
commissions attributable to the sale of Registrable Securities.

         "SELLING HOLDER" means any Holder who sells Registrable Securities
pursuant to Section 2.01 or 2.02.

         "SERIES A PREFERRED SHARES" means shares of Series A Convertible
Preferred Stock, par value $0.01 per share, of the Issuer.

                                       2
<PAGE>   5



         "SERIES B PREFERRED SHARES" means shares of Series B Convertible
Preferred Stock, par value $0.01 per share, of the Issuer.

         "SECURITIES ACT" means the Securities Act of 1933, as amended, and the
rules and regulations promulgated thereunder.

         "SUBSIDIARY" means any entity of which ownership interests having
ordinary voting power to elect a majority of the board of directors or other
Persons performing similar functions are at the time directly or indirectly
owned by the Issuer.

          (b) Each of the following terms is defined in the Section set forth
opposite such term:

<TABLE>
<CAPTION>
                TERM                                SECTION
                ----                                -------

<S>                                               <C>
Holders                                             Preamble
Indemnified Party                                   2.06(c)
Indemnifying Party                                  2.06(c)
Inspectors                                          2.03(h)
Issuer                                              Preamble
Purchase Agreement                                  Recitals
Records                                             2.03(h)
</TABLE>


                                    ARTICLE 2

                               REGISTRATION RIGHTS

         SECTION 2.01. Demand Registration Rights. (a) Each of MS and Provender
shall have the right to make written demand upon the Issuer, on not more than
two separate occasions (subject to the provisions of this Section 2.01), to
register under the Securities Act outstanding Registrable Securities, and the
Issuer shall use commercially reasonable efforts to cause such shares to be
registered under the Securities Act as soon as reasonably practicable so as to
permit the sale thereof promptly; provided that each such demand shall cover, as
the case may be, at least (i) $500,000 liquidation preference of Series A
Preferred Shares or Series B Preferred Shares, as the case may be, or (ii)
41,667 Common Shares (subject in each such case to adjustment for stock splits,
reverse stock splits, stock dividends and similar events after the date hereof).
Each such demand will specify the type and number of Registrable Securities
proposed to be sold and the intended method of disposition thereof. A
registration that is a Public Offering shall not count for purposes of
determining the number of registrations to which the requesting Holder is
entitled pursuant to this Section 2.01(a) unless the

                                       3
<PAGE>   6



requesting Holder is able to register and sell at least 75% of the Registrable
Securities requested to be included in such registration.

          (b) Notwithstanding the provisions of Section 2.01(a), the Issuer (i)
shall not be obligated to prepare or file more than one registration statement
pursuant to this Section 2.01 during any 9-month period and (ii) shall be
entitled to postpone the filing of any registration statement otherwise required
to be prepared and filed pursuant to Section 2.01(a) for a period of up to 90
days if the Issuer determines in its reasonable judgment and in good faith that
the registration and distribution of the Registrable Securities that are the
subject of such registration would impair or interfere with in any material
respect any contemplated financing, acquisition, disposition, corporate
reorganization or other material transaction or corporate development involving
the Issuer or any of its Subsidiaries or would require premature disclosure
thereof. In the event of such postponement, the requesting Holder shall have the
right to withdraw the request for registration by giving written notice to the
Issuer within 20 days after receipt of the notice of postponement and, in the
event of such withdrawal, such request shall not be counted for purposes of
determining the number of registrations to which the requesting Holder is
entitled pursuant to Section 2.01(a).

          (c) The requesting Holder shall be entitled to select the managing
underwriter or underwriters of any Public Offering effected pursuant to this
Section 2.01, which selection shall be reasonably satisfactory to the Issuer.
The Issuer will enter into customary agreements (including an underwriting
agreement in customary form) and take such other actions as are reasonably
required in order to expedite or facilitate the disposition of the Registrable
Securities to be sold pursuant thereto.

         SECTION 2.02. Piggy-Back Registration Rights. (a) If the Issuer at any
time proposes to register any of its Common Shares, Series A Preferred Shares or
Series B Preferred Shares under the Securities Act (other than (i) by a
registration on Form S-4, Form S-8 or any successor or similar form, (ii)
pursuant to a stock option or other employee benefit or similar plan or (iii)
pursuant to a direct share purchase plan, dividend reinvestment plan or similar
plan of the Issuer), the Issuer shall, as promptly as practicable, give written
notice to each Holder of the Issuer's intention to effect such registration. If,
within 15 days after receipt of such notice, any Holder submits a written
request to the Issuer specifying the number of Registrable Securities intended
to be disposed of by such Holder, the Issuer will use commercially reasonable
efforts to include such the shares specified in such Holder's request in such
registration. No registration effected under this Section 2.02 shall relieve the
Issuer of its obligation to effect any registration upon request under Section
2.01. If a Holder has been permitted to participate in a proposed offering
pursuant to this Section 2.02(a), the Issuer thereafter may determine either not
to file a registration relating thereto, or to withdraw such registration
statement, or otherwise not consummate such offering, without any liability

                                       4
<PAGE>   7



hereunder other than its obligation to pay the Registration Expenses in
connection therewith.

          (b) If a registration pursuant to this Section 2.02 involves a Public
Offering and the managing underwriter shall advise the Issuer that the inclusion
of all the securities proposed to be included in such registration would exceed
the largest number of securities that can be sold without having a material
adverse effect on such offering, including the price at which such securities
can be sold, the Issuer will include in such registration (i) first, if the
registration is initiated by the Issuer, the Common Shares the Issuer proposes
to sell and, if the registration is initiated by a Person (other than the Issuer
or a Holder), the Common Shares such Person proposes to sell, (ii) second, the
Registrable Securities requested to be included in such registration by the
Holders, allocated (if necessary) among such Holders pro rata based on the
Registrable Securities requested to be included in such registration, and (iii)
third, if the registration is initiated by the Issuer, Common Shares to be sold
for the account of other Persons having incidental registration rights and, if
the registration is initiated by a Person (other than the Issuer), the Common
Shares to be sold for the account of the Issuer and the Common Shares to be sold
for the account of other Persons having incidental registration rights, in each
such case, with such priorities among them as the Issuer shall determine.

         SECTION 2.03. Registration Procedures. If the Issuer is required to use
commercially reasonable efforts to effect the registration of any Registrable
Securities under the Securities Act as provided in Section 2.01 or 2.02, the
Issuer will, as promptly as possible:

          (a) prepare and file with the Commission a registration statement on
an appropriate form, and thereafter use commercially reasonable efforts to cause
such registration statement to become effective and to remain effective for 90
days or such shorter period as shall be necessary to effect the distribution of
the Registrable Securities and prepare and file with the Commission such
amendments and supplements to such registration statement and the prospectus
used in connection therewith as may be necessary to keep such registration
statement effective for the period specified above and to comply with the
provisions of the Securities Act with respect to the disposition of all
securities covered by such registration statement until such time as all of such
securities have been disposed of in accordance with the intended methods of
disposition by the Selling Holders set forth in such registration statement;
provided that the Issuer will, at least 5 Business Days (or at least 3 Business
Days in the case of incidental registrations) prior to filing a registration
statement or prospectus or any amendment or supplement thereto, furnish to the
Selling Holders copies of such registration statement or prospectus (or
amendment or supplement) as proposed to be filed (including, upon the request of
any Selling Holder, documents to be incorporated by reference therein) which
documents will be subject to the

                                       5
<PAGE>   8



reasonable review and comments of the Selling Holders (and their attorneys)
during such 5-Business Day period (or 3-Business Day period, as the case may be)
and the Issuer will not file any registration statement, any prospectus or any
amendment or supplement thereto (or any such documents incorporated by
reference) containing any statements with respect to the Selling Holders to
which the Selling Holders shall reasonably object in writing;

          (b) furnish to the Selling Holders and to any underwriter such number
of conformed copies of such registration statement and of each such amendment
and supplement thereto (in each case including all exhibits), the prospectus
contained in such registration statement (including each preliminary prospectus
and any summary prospectus) and any other prospectus filed under Rule 424 or
Rule 430A under the Securities Act, in conformity with the requirements of the
Securities Act, documents incorporated by reference in such registration
statement, amendment, supplement or prospectus and such other documents (in each
case including all exhibits), as the Selling Holders or such underwriter may
reasonably request;

          (c) after the filing of the registration statement, promptly notify
the Selling Holders of the effectiveness thereof and of any stop order issued or
threatened by the Commission and take all reasonable actions required to prevent
the entry of such stop order or to remove it if entered and promptly notify the
Selling Holders of such lifting or withdrawal of such order;

          (d) use commercially reasonable efforts to register or qualify all
Registrable Securities and other securities covered by such registration
statement under such other securities or blue sky laws of such jurisdictions as
the Selling Holders or any underwriter shall reasonably request, to keep such
registration or qualification in effect for so long as such registration
statement remains in effect, and take any other action which may be reasonably
necessary or advisable to enable the Selling Holders to consummate the
disposition in such jurisdictions of the securities owned by the Selling
Holders, except that the Issuer shall not for any such purpose be required to
qualify generally to do business as a foreign corporation in any jurisdiction
wherein it would not but for the requirements of this Section 2.03(d) be
obligated to be so qualified, to subject itself to taxation in any such
jurisdiction or to consent to general service of process in any such
jurisdiction;

          (e) use commercially reasonable efforts to cause all Registrable
Securities covered by such registration statement to be registered with or
approved by such other governmental agencies or authorities as may be necessary
to enable the Selling Holders to consummate the disposition of such Registrable
Securities;

          (f) furnish to the Selling Holders and to each underwriter, if any, a
signed counterpart of: (i) an opinion of counsel for the Issuer addressed to the

                                       6
<PAGE>   9



Selling Holders and such underwriter on which opinion the Selling Holders and
such underwriter are entitled to rely and (ii) a "comfort" letter signed by the
independent public accountants who have certified the Issuer's financial
statements included in such registration statement, each in customary form and
covering such matters of the type customarily covered by opinions or comfort
letters, as the case may be, as the Selling Holders or the managing underwriter
therefor reasonably request. The Issuer will use its best efforts to have such
comfort letters addressed to each Selling Holder;

          (g) immediately notify the Selling Holders at any time when a
prospectus relating thereto is required to be delivered under the Securities
Act, upon discovery that, or upon the happening of any event as a result of
which, the prospectus included in such registration statement, as then in
effect, includes an untrue statement of a material fact or omits to state any
material fact required to be stated therein or necessary to make the statements
therein not misleading in the light of the circumstances under which they were
made, and promptly prepare and furnish to the Selling Holders a reasonable
number of copies of any supplement to or amendment of such prospectus as may be
necessary so that, as thereafter delivered to the purchasers of such securities,
such prospectus shall not include an untrue statement of a material fact or omit
to state a material fact required to be stated therein or necessary to make
statements therein not misleading in the light of the circumstances under which
they were made;

          (h) make available for inspection by the Selling Holders, any
underwriter participating in any disposition pursuant to such registration
statement and any attorney, accountant or other professional retained by any
Holder or underwriter (collectively, the "INSPECTORS"), all financial and other
records, pertinent corporate documents and properties of the Issuer
(collectively, the "RECORDS") as shall be reasonably necessary to enable them to
exercise their due diligence responsibility, and shall cause (i) the Issuer's
officers, directors and employees to supply all information reasonably requested
by any Inspectors and (ii) the senior management of the Issuer and its
Subsidiaries to participate in any "road show" presentations to investors for
such period of time as is reasonably requested by the managing underwriters, in
each case in connection with such registration statement. Each Selling Holder
agrees that information obtained by it as a result of such inspections shall be
deemed confidential and shall not be used by it as the basis for any market
transactions in the securities of the Issuer unless and until such information
is made generally available to the public. Each Selling Holder further agrees
that it will, upon learning that disclosure of such Records is sought in a court
of competent jurisdiction, give notice to the Issuer and allow the Issuer, at
its expense, to undertake appropriate action to prevent disclosure of the
Records deemed confidential;

                                       7
<PAGE>   10



          (i) use commercially reasonable efforts to list all Registrable
Securities covered by such registration statement on any securities exchange or
quotation system on which any of the Common Shares is then listed or traded; and

          (j) otherwise use commercially reasonable efforts to comply with all
applicable rules and regulations of the Commission, and make available to its
security holders, as soon as reasonably practicable, an earnings statement or
such other document that shall satisfy the provisions of Section 11(a) of the
Securities Act and Rule 158 thereunder.

         The Issuer may require each Selling Holder to promptly furnish to the
Issuer, as a condition precedent to including such Holder's Registrable
Securities in any registration, such written information regarding such Holder
and the distribution of such securities as the Issuer may from time to time
reasonably request in writing.

         Each Selling Holder agrees that upon receipt of any notice from the
Issuer of the happening of any event of the kind described in Section 2.03(g),
such Holder will forthwith discontinue its disposition of Registrable Securities
pursuant to the registration statement relating to such Registrable Securities
until such Holder's receipt of the copies of the supplemented or amended
prospectus contemplated by Section 2.03(g) and, if so directed by the Issuer,
will deliver to the Issuer (at the Issuer's expense) all copies, other than
permanent file copies, then in such Holder's possession, of the prospectus and
any amendments or supplements thereto relating to such Registrable Securities
current at the time of receipt of such notice. In the event the Issuer shall
give such notice, the Issuer shall extend the period during which the
effectiveness of such registration statement shall be maintained by the number
of days during the period from and including the date of the giving of notice
pursuant to Section 2.03(g) to the date when the Issuer shall make available to
the Selling Holders a prospectus supplemented or amended to conform with the
requirements of Section 2.03(g).

         The Issuer will promptly pay all Registration Expenses in connection
with any registration of Registrable Securities pursuant to Section 2.01 or
2.02.

         SECTION 2.04. Participation in Underwritten Registrations. In
connection with any Public Offering pursuant to Section 2.01 or 2.02, each
Selling Holder agrees to complete and execute all questionnaires, powers of
attorney, indemnities, underwriting agreements and other documents reasonably
required under the terms of the relevant underwriting arrangements.

         SECTION 2.05. Holdback Agreements. If any registration or offering of
Registrable Securities shall be in connection with a Public Offering, the Issuer
and each Selling Holder agree not to effect any public sale or distribution of
any Common Shares or any securities convertible into or exchangeable or
exercisable
                                       8

<PAGE>   11



for Common Shares (in each case other than as part of such Public Offering), if
and to the extent requested by the managing underwriter during the 90-day period
beginning on the effective date of such registration statement without the
written consent of such managing underwriter.

         SECTION 2.06. Indemnification. (a) Indemnification by the Issuer. The
Issuer agrees to indemnify and hold harmless each Selling Holder, its officers,
directors and agents and each Person, if any, who controls each Selling Holder
within the meaning of Section 15 of the Securities Act or Section 20 of the
Exchange Act from and against any and all losses, claims, damages, liabilities
or expenses caused by any untrue statement or alleged untrue statement of a
material fact contained in any registration statement or prospectus relating to
the Registrable Securities (as amended or supplemented if the Issuer shall have
furnished any amendments or supplements thereto) or caused by any omission or
alleged omission to state therein a material fact required to be stated therein
or necessary to make the statements therein not misleading, and the Issuer will
reimburse each Selling Holder for any legal or any other expenses reasonably
incurred by them in connection with investigating or defending such loss, claim,
damage, liability or expense, except insofar as such losses, claims, damages,
liabilities or expenses are caused by any such untrue statement or omission or
alleged untrue statement or omission based upon information furnished in writing
to the Issuer by such Selling Holder or on such Selling Holder's behalf
expressly for use therein; provided that with respect to any untrue statement or
omission or alleged untrue statement or omission made in any preliminary
prospectus, or in any prospectus, as the case may be, the indemnity agreement
contained in this paragraph shall not apply to the extent that any such loss,
claim, damage, liability or expense results from the fact that a current copy of
the prospectus (or the amended or supplemented prospectus, as the case may be)
was not sent or given to the Person asserting any such loss, claim, damage,
liability or expense at or prior to the written confirmation of the sale of the
Registrable Securities concerned to such Person if it is determined that the
Issuer has provided such prospectus (or amended or supplemented prospectus) and
it was the responsibility of such Selling Holder to provide such Person with a
current copy of the prospectus (or such amended or supplemented prospectus, as
the case may be) and such current copy of the prospectus (or such amended or
supplemented prospectus, as the case may be) would have cured the defect giving
rise to such loss, claim, damage, liability or expense. The Issuer also agrees
to indemnify any underwriters of the Registrable Securities, their officers and
directors and each Person who controls such underwriters on substantially the
same basis as that of the indemnification of each Selling Holder provided in
this Section 2.06(a).

          (b) Indemnification by the Holder. Each Selling Holder agrees,
severally but not jointly, to indemnify and hold harmless the Issuer, its
officers, directors and agents and each Person, if any, who controls the Issuer
within the meaning of either Section 15 of the Securities Act or Section 20 of
the Exchange

                                       9
<PAGE>   12



Act to the same extent as the foregoing indemnity from the Issuer to each
Selling Holder, but only (i) with respect to information furnished in writing by
such Selling Holder or on such Selling Holder's behalf expressly for use in any
registration statement or prospectus relating to the Registrable Securities, or
any amendment or supplement thereto, or any preliminary prospectus or (ii) to
the extent that any loss, claim, damage, liability or expense described in
Section 2.06(a) results from the fact that a current copy of the prospectus (or
the amended or supplemented prospectus, as the case may be) was not sent or
given to the Person asserting any such loss, claim, damage, liability or expense
at or prior to the written confirmation of the sale of the Registrable
Securities concerned to such Person if it is determined that it was the
responsibility of such Selling Holder to provide such Person with a current copy
of the prospectus (or such amended or supplemented prospectus, as the case may
be) and such current copy of the prospectus (or such amended or supplemented
prospectus, as the case may be) would have cured the defect giving rise to such
loss, claim, damage, liability or expense. Each Selling Holder also agrees to
indemnify and hold harmless the underwriters of the Registrable Securities,
their officers and directors and each Person who controls such underwriters on
substantially the same basis as that of the indemnification of the Issuer
provided in this Section 2.06(b).

          (c) Conduct of Indemnification Proceedings. In case any proceeding
(including any governmental investigation) shall be instituted involving any
Person in respect of which indemnity may be sought pursuant to this Section
2.06, such Person (an "INDEMNIFIED PARTY") shall promptly notify the Person
against whom such indemnity may be sought (the "INDEMNIFYING PARTY") in writing
and the Indemnifying Party shall assume the defense thereof, including the
employment of counsel reasonably satisfactory to such Indemnified Party, and
shall assume the payment of all fees and expenses; provided that the failure of
any Indemnified Party so to notify the Indemnifying Party shall not relieve the
Indemnifying Party of its obligations hereunder except to the extent that the
Indemnifying Party is materially prejudiced by such failure to notify. In any
such proceeding, any Indemnified Party shall have the right to retain its own
counsel, but the fees and expenses of such counsel shall be at the expense of
such Indemnified Party unless (i) the Indemnifying Party and the Indemnified
Party shall have mutually agreed to the retention of such counsel or (ii) in the
reasonable judgment of such Indemnified Party representation of both parties by
the same counsel would be inappropriate due to actual or potential differing
interests between them. It is understood that the Indemnifying Party shall not,
in connection with any proceeding or related proceedings in the same
jurisdiction, be liable for the reasonable fees and expenses of more than one
separate firm of attorneys (in addition to any local counsel) at any time for
all such Indemnified Parties, and that all such fees and expenses shall be
reimbursed as they are incurred. In the case of any such separate firm for the
Indemnified Parties, such firm shall be designated in writing by the Indemnified
Party that had the largest number of Registrable Securities included in such
registration. The Indemnifying

                                       10
<PAGE>   13



Party shall not be liable for any settlement of any proceeding effected without
its written consent, but if settled with such consent, or if there be a final
judgment for the plaintiff, the Indemnifying Party shall indemnify and hold
harmless such Indemnified Parties from and against any loss or liability (to the
extent stated above) by reason of such settlement or judgment. No Indemnifying
Party shall, without the prior written consent of the Indemnified Party, effect
any settlement of any pending or threatened proceeding in respect of which any
Indemnified Party is or could have been a party and indemnity could have been
sought hereunder by such Indemnified Party, unless such settlement includes an
unconditional release of such Indemnified Party from all liability arising out
of such proceeding.

          (d) Contribution. If the indemnification provided for in this Section
2.06 is unavailable to the Indemnified Parties in respect of any losses, claims,
damages or liabilities referred to herein, then each Indemnifying Party, in lieu
of indemnifying such Indemnified Party, shall contribute to the amount paid or
payable by such Indemnified Party as a result of such losses, claims, damages or
liabilities (i) as between the Issuer and the Selling Holders on the one hand
and the underwriters on the other, in such proportion as is appropriate to
reflect the relative benefits received by the Issuer and the Selling Holders on
the one hand and the underwriters on the other, from the offering of the
Registrable Securities, or if such allocation is not permitted by applicable
law, in such proportion as is appropriate to reflect not only the relative
benefits but also the relative fault of the Issuer and the Selling Holders on
the one hand and of such underwriters on the other in connection with the
statements or omissions which resulted in such losses, claims, damages or
liabilities, as well as any other relevant equitable considerations and (ii) as
between the Issuer on the one hand and the Selling Holders on the other, in such
proportion as is appropriate to reflect the relative fault of the Issuer and of
the Selling Holders in connection with such statements or omissions, as well as
any other relevant equitable considerations. The relative benefits received by
the Issuer and the Selling Holders on the one hand and such underwriters on the
other shall be deemed to be in the same proportion as the total proceeds from
the offering (net of underwriting discounts and commissions but before deducting
expenses) received by the Issuer and the Selling Holders bear to the total
underwriting discounts and commissions received by such underwriters, in each
case as set forth in the table on the cover page of the prospectus. The relative
fault of the Issuer and the Selling Holders on the one hand and of such
underwriters on the other shall be determined by reference to, among other
things, whether the untrue or alleged untrue statement of a material fact or the
omission or alleged omission to state a material fact relates to information
supplied by the Issuer and a Selling Holder or by such underwriters. The
relative fault of the Issuer on the one hand and of the Selling Holders on the
other shall be determined by reference to, among other things, whether the
untrue or alleged untrue statement of a material fact or the omission or alleged
omission to state a material fact relates to information supplied by such party,
and the parties' relative intent,

                                       11
<PAGE>   14



knowledge, access to information and opportunity to correct or prevent such
statement or omission.

         The Issuer and the Selling Holders agree that it would not be just and
equitable if contribution pursuant to this Section 2.06 were determined by pro
rata allocation (even if the underwriters were treated as one entity for such
purpose) or by any other method of allocation which does not take account of the
equitable considerations referred to in the immediately preceding paragraph. The
amount paid or payable by an Indemnified Party as a result of the losses,
claims, damages or liabilities referred to in the immediately preceding
paragraph shall be deemed to include, subject to the limitations set forth
above, any legal or other expenses reasonably incurred by such Indemnified Party
in connection with investigating or defending any such action or claim.
Notwithstanding the provisions of this Section 2.06, no underwriter shall be
required to contribute any amount in excess of the amount by which the total
price at which the Registrable Securities underwritten by it and distributed to
the public were offered to the public exceeds the aggregate amount of any
damages which such underwriter has otherwise been required to pay by reason of
such untrue or alleged untrue statement or omission or alleged omission, and no
Selling Holder shall be required to contribute any amount in excess of the
amount by which the total price at which the Registrable Securities of such
Selling Holder were offered to the public exceeds the amount of any damages
which such Selling Holder has otherwise been required to pay by reason of such
untrue or alleged untrue statement or omission or alleged omission. No Person
guilty of fraudulent misrepresentation (within the meaning of Section 11(f) of
the Securities Act) shall be entitled to contribution from any Person who was
not guilty of such fraudulent misrepresentation.

                                    ARTICLE 3

                                  MISCELLANEOUS

         SECTION 3.01. Notices. All notices, requests and other communications
to any party hereunder shall be in writing (including telecopier or similar
writing) and shall be given to such party at its address or telecopier number
set forth on the signature page hereof, or such other address or telecopier
number as such party may hereinafter specify for the purpose to the party giving
such notice. Each such notice, request or other communication shall be effective
(i) if given by telecopy, when such telecopy is transmitted to the telecopy
number specified pursuant to this Section 3.01 and the appropriate communication
is received, (ii) if given by mail, 72 hours after such communication is
deposited in the mails with first class postage prepaid, addressed as aforesaid
or (iii) if given by any other means, when delivered at the address specified in
this Section 3.01.

                                       12
<PAGE>   15



         SECTION 3.02. Amendments; Waivers. (a) No failure or delay on the part
of any party in exercising any right, power or privilege hereunder shall operate
as a waiver thereof, nor shall any single or partial exercise thereof preclude
any other or further exercise thereof or the exercise of any other right, power
or privilege. The rights and remedies herein provided shall be cumulative and
not exclusive of any rights or remedies provided by law.

          (b) Any provision of this Agreement may be amended or waived if, but
only if, such amendment or waiver is in writing and is signed, in the case of an
amendment, by the parties hereto, or in the case of a waiver, by the party
against whom the waiver is to be effective.

         SECTION 3.03. Successors; Assigns. The provisions of this Agreement
shall be binding upon and accrue to the benefit of the parties hereto and their
respective successors and permitted assigns; provided that no party may assign
any of its rights or obligations under this Agreement without the other party's
prior written consent. Notwithstanding the foregoing, if MS or Provender
transfers to any Person more than 50% of the Series A Preferred Shares or Series
B Preferred Shares, as the case may be, acquired by it pursuant to the Purchase
Agreement, such Holder may assign its rights under Article 2 to such Person
without the prior written consent of the Issuer; provided that such Person
agrees to be bound by the provisions of this Agreement as if such Person were
such Holder hereunder.

         SECTION 3.04. Entire Agreement. The Purchase Agreement (including the
Disclosure Schedule thereto) and this Agreement constitute the entire agreement
and understanding of the parties hereto and supersede any and all prior
agreements and understandings between the parties hereto, written or oral,
relating to the subject matter hereof.

         SECTION 3.05.  Applicable Law.  This Agreement shall be construed in
accordance with and governed by the laws of the State of New York, without
regard to the conflicts of law rules of such state.

         SECTION 3.06. Remedies. The parties hereto acknowledge and agree that
in the event of any breach of this Agreement, the parties would be irreparably
harmed and could not be made whole by monetary damages. Each party hereto
accordingly agrees (i) not to assert by way of defense or otherwise that a
remedy at law would be adequate, and (ii) that the parties agree, in addition to
any other remedy to which they may be entitled, that the remedy of specific
performance of this Agreement is appropriate in any action in court.

         SECTION 3.07.  Severability.  The invalidity or unenforceability of any
provisions of this Agreement in any jurisdiction shall not affect the validity,
legality or enforceability of the remainder of this Agreement in such
jurisdiction

                                       13
<PAGE>   16



or the validity, legality or enforceability of this Agreement, including any
such provision, in any other jurisdiction, it being intended that all rights and
obligations of the parties hereunder shall be enforceable to the fullest extent
permitted by law.

         SECTION 3.08. Fees and Expenses. Unless otherwise provided herein, all
costs and expenses incurred in connection with the transactions contemplated by
this Agreement shall be paid by the party incurring such costs and expenses.

         SECTION 3.09. Counterparts. This Agreement may be executed in any
number of counterparts, each of which shall be an original with the same effect
as if the signatures thereto and hereto were upon the same instrument.

                                       14
<PAGE>   17



         IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be
duly executed as of the date first above written.

                              CARVER BANCORP, INC.

                              By:     /s/ Deborah C.  Wright
                                     -----------------------------------------
                                     Name: Deborah C. Wright
                                     Title: President and Chief Executive
                                              Officer

                              Address for notices:

                                     Carver Bancorp, Inc.
                                     75 West 125th Street
                                     New York, New York 10027
                                     Facsimile: (212) 426-6214
                                     Attention: President and Chief Executive
                                                  Officer

                              MORGAN STANLEY & CO. INCORPORATED

                              By:     /s/ Roxanne M. Beer
                                     -----------------------------------------
                                     Name: Roxanne M. Beer
                                     Title: Vice President

                              Address for notices:

                                     Morgan Stanley & Co. Incorporated
                                     1585 Broadway
                                     New York, NY 10036
                                     Facsimile: 212-761-0358
                                     Attention: Roxanne M. Beer


<PAGE>   18


                              PROVENDER OPPORTUNITIES FUND L.P.

                              By:     /s/ Derek K. Jones
                                     -----------------------------------------
                                     Name: Derek K. Jones
                                     Title:   General Partner

                              By:     /s/ Raymond J. Walsh, Jr.
                                     -----------------------------------------
                                     Name: Raymond J. Walsh, Jr.
                                     Title:   General Partner

                              Address for notices:

                                     Provender Capital Group, LLC
                                     17 State Street
                                     New York, New York 10004
                                     Facsimile: (212) 271-8875
                                     Attention: Raymond J. Walsh, Jr.