<PAGE>

                                                                     EXHIBIT 4.3

                            BMO NESBITT BURNS CORP.
                            111 West  Monroe Street
                            Chicago, Illinois 60603

                                BANK OF MONTREAL
                             115 S.  LaSalle Street
                            Chicago, Illinois 60603

                        First Amendment to Waiver Letter

                                                                 January 8, 2001

TO:  Imperial Securitization Corporation
     PO Box 9
     8016 Highway 90A
     Sugar Land, TX 77478

     Imperial Distributing, Inc.
     PO Box 9
     8016 Highway 90A
     Sugar Land, TX 77478

     Imperial Sugar Company
     PO Box 9
     8016 Highway 90A
     Sugar Land, TX 77478

     Re:  Imperial Securitization Corporation

Ladies and Gentlemen:

     We refer to the Waiver Letter between us dated as of December 13, 2000,
with an effective date of September 30, 2000 (the "Waiver Letter"), capitalized
terms used without definition below to have the meanings ascribed to them in the
Waiver Letter.  Upon receipt by the Agent of counterparts hereof which, taken
together, bear the signatures of the Imperial Securitization Corporation,
Imperial Distributing, Inc., Imperial Sugar Company, the Agent and the Bank of
Montreal, the Waiver Letter shall be amended by:

     (i) striking the date "January 8, 2001" in the fourth paragraph thereof and
     substituting the following "the earlier of January 16, 2001 and the
     expiration of the waiver of the Default or Event of Default under the
     Credit Agreement, as provided in the Interim Waiver
<PAGE>

     Agreement dated as of September 30, 2000 and as amended by the First
     Amendment to the Interim Waiver Agreement dated as of January 8, 2001"
     therefore and

     (ii) striking the date "January 8, 2001" in the fifth paragraph thereof and
     substituting the date "January 16, 2001" therefore.

     Except as specifically amended hereby all of the terms, conditions and
provisions of the Waiver Letter shall stand and remain unchanged and in full
force and effect.  No reference to this First Amendment to Waiver Letter need be
made in any instrument or document at any time referring to the Waiver Letter, a
reference to the Waiver Letter in any of such to be deemed to be a reference to
the same as amended hereby.  This First Amendment to Waiver Letter may be
executed in counterparts and by separate parties hereto on separate
counterparts, each to constitute an original but all of which shall constitute
one and the same instrument.  The amendments provided for herein shall be
strictly construed and limited as herein provided.  This First Amendment to the
Waiver Letter shall be construed in accordance with and governed by the laws of
the state of New York.

     Dated and to be come effective as of the 8th day of January 2001.
<PAGE>

                              Very truly yours,

                              BMO NESBITT BURNS CORP.


                              By: /s/ DAVID J. KUCERA
                                 -------------------------------
                              Printed Name: David J. Kucera
                              Title:  Managing Director


                              By: /s/ JAMES P. WALSH
                                 -------------------------------
                              Printed Name: James P. Walsh
                              Title: Managing Director


                              BANK OF MONTREAL


                              By: /s/ GEOFFREY R. McCONNELL
                                 --------------------------------
                              Printed Name: Geoffrey R. McConnell
                              Title: Director


                                      S-1
<PAGE>

Acknowledged and Agreed:

IMPERIAL SECURITIZATION CORPORATION

By: /s/ KAREN L. MERCER
   ---------------------------------
Printed Name: Karen L. Mercer
Title: Treasurer

IMPERIAL DISTRIBUTING, INC.

By: /s/ KAREN L. MERCER
   ---------------------------------
Printed Name: Karen L. Mercer
Title: Treasurer

IMPERIAL SUGAR COMPANY

By: /s/ KAREN L. MERCER
  ----------------------------------
Printed Name: Karen L. Mercer
Title: VP and Treasurer



                                      S-2