<PAGE>
 
                                                        EXHIBIT 4.6

      AMENDMENT No. 5 to Pooling and Servicing Agreement

     AMENDMENT No. 5 to the Master Pooling and Servicing Agreement, dated as
of June 30, 1992 (the "Agreement"), as amended by Amendment No. 1 to Pooling
and Servicing Agreement, dated as of September 8, 1993, Amendment No. 2 to
Pooling and Servicing Agreement, dated as of July 15, 1994, by Amendment No.
3 to Pooling and Servicing Agreement, dated as of September 12, 1994, and by
Amendment No. 4 to Pooling and Servicing Agreement, dated as of January 17,
1996, and as amended by certain amendments prior to the date hereof to the
Series Supplements thereunder, each between AMERICAN EXPRESS RECEIVABLES
FINANCING CORPORATION, a Delaware corporation, and AMERICAN EXPRESS TRAVEL
RELATED SERVICES COMPANY, INC., a corporation organized under the laws of the
State of New York, as Servicer, and THE BANK OF NEW YORK, a banking corporation
organized under the laws of New York, as Trustee.  Capitalized terms used
herein and not defined herein have the respective meanings assigned to them in
the Agreement.


                     W I T N E S S E T H:

     WHEREAS, pursuant to Section 13.01(a) of the Agreement, the Agreement and
any Supplement thereto may be amended from time to time by the Servicer, the
Transferor and the Trustee, without the consent of any of the Investor
Certificateholders, among other things, to cure any ambiguity, to correct or
supplement any provisions therein which may be inconsistent with any other
provisions therein or to add any other provisions with respect to matters or
questions raised under the Agreement which shall not be inconsistent with the
provisions of the Agreement; provided however that such action shall not
adversely affect in any material respect the interests of any of the Investor
Certificateholders.   Prior to executing any such amendment in accordance with
subsection 13.01(a), the Trustee shall receive and shall be permitted to rely
upon an Opinion of Counsel to the effect that the conditions and requirements
of subsection 13.01(a) have been satisfied. 

     WHEREAS, the Servicer and the Transferor consider it is necessary to
amend the Agreement as set forth herein in accordance with Section 13.01(a) to
make clear that in accordance with the Account Guidelines, Recoveries may
include net proceeds of sales of Receivables in Defaulted Accounts which are
entered into for purposes of collecting on such Receivables.

     WHEREAS, the Trustee is willing to accept such amendments to the
Agreement, subject to the terms and conditions thereof;

     NOW THEREFORE, the Servicer, the Transferor and the Trustee agree as
follows:

     1.  The definition of "Receivable" contained in the Agreement is hereby
amended to add the following sentence at the end thereof:

     "Receivables in a Defaulted Account will cease to be included as
receivables at such time as they are sold as part of the Servicer's collection
efforts."




<PAGE>
     2.  The definition of "Recoveries" is hereby amended to read as follows:

     "Recoveries" shall mean all amounts received with respect to Receivables
in Defaulted Accounts, net of expenses allocable thereto, including the net
proceeds of any sale of such Receivables."
         
     3.  This Amendment shall be effective as of March 1, 1997.

     4.  The Trustee acknowledges that it has received an Opinion of Counsel
pursuant to Section 13.01(g) of the Agreement, a copy of which is attached
hereto, to the effect that the conditions and requirements of Section 13.01(a)
of the Agreement have been satisfied.  

     IN WITNESS WHEREOF, the parties have caused this Amendment to be duly
executed and delivered by their respective duly authorized officers as of the
27th day of March 1997.


                                     AMERICAN EXPRESS RECEIVABLES
                                     FINANCING CORPORATION 
                                     as Transferor

   
                                     By:  /s/ Leslie R. Scharfstein
                                           ------------------------ 
                                     Name:  Leslie R. Scharfstein 
                                     Title: Vice President


                                     AMERICAN EXPRESS TRAVEL RELATED
                                     SERVICES COMPANY, INC.
                                     as Servicer


                                     By:  /s/ Jay  B. Stevelman
                                          ---------------------
                                     Name:  Jay B. Stevelman
                                     Title: Treasurer & SVP


                                     THE BANK OF NEW YORK,
                                     as Trustee and Paying Agent


                                     By:  /s/  Reyne A. Macadaeg
                                         -----------------------
                                     Name:  Reyne A. Macadaeg
                                     Title: Assistant Vice President