<PAGE>   1
                                                                     EXHIBIT 4.3


                SECOND CONSENT AND WAIVER UNDER CREDIT AGREEMENT


         THIS SECOND CONSENT AND WAIVER UNDER CREDIT AGREEMENT (this "Consent
and Waiver") is dated as of the 12th day of November, 1997 among SAVANNAH FOODS
& INDUSTRIES, INC. (the "Borrower"), WACHOVIA BANK, N.A. (successor by merger to
Wachovia Bank of Georgia, N.A.), as Agent (the "Agent") and WACHOVIA BANK, N.A.,
THE CHASE MANHATTAN BANK, THE FIRST NATIONAL BANK OF CHICAGO, CITIZENS BANK
(formerly Second National Bank of Saginaw), SUNTRUST BANK, ATLANTA, and
COOPERATIEVE CENTRALE RAIFFEISEN-BOERENLEEENBANK B.A., "RABOBANK NEDERLAND", NEW
YORK BRANCH (collectively, the "Banks");

                                   WITNESSETH:

         WHEREAS, the Borrower, the Agent and the Banks executed and delivered
that certain Credit Agreement, dated as of April 1, 1996, as amended by the
Consent and Waiver under and First Amendment to Credit Agreement, dated as of
October 16, 1997 (as so amended, the "Credit Agreement");

         WHEREAS, the Borrower has requested that the Agent and the Banks
consent to a certain transfer of funds from the Borrower to IHC which would not
be permitted under Section 6.05, 6.16, 6.17 or 6.25, as applicable, of the
Credit Agreement and waive any Event of Default which otherwise would occur
under Section 6.05, 6.16, 6.17 or 6.25 as a result thereof, and the Agent and
the Banks have agreed to such consent and waiver, subject to the terms and
conditions hereof;

         NOW, THEREFORE, for and in consideration of the above premises and
other good and valuable consideration, the receipt and sufficiency of which
hereby is acknowledged by the parties hereto, the Borrower, the Agent and the
Banks hereby covenant and agree as follows:

         1. Definitions. Unless otherwise specifically defined herein, each term
used herein which is defined in the Credit Agreement shall have the meaning
assigned to such term in the Credit Agreement. Each reference to "hereof",
"hereunder", "herein" and "hereby" and each other similar reference and each
reference to "this Agreement" and each other similar reference contained in the
Credit Agreement shall from and after the date hereof refer to the Credit
Agreement as amended hereby.




                                      68
<PAGE>   2




         2. Consent and Waiver. IHC is making a tender offer for Capital Stock
of the Borrower, which tender offer is more particularly described in the LCPI
Commitment (the "IHC Tender Offer"). The Borrower has proposed that, if the IHC
Tender Offer is successful, in connection therewith, on or after the IHC Tender
Date, the following actions be taken (collectively, the "Flexitrust Tender
Proceeds Actions): (i) IHC would, for a purchase price of approximately
$29,900,000, buy approximately 1,475,000 of the 2,500,000 shares of Capital
Stock of the Borrower held by the Flexitrust (a benefits trust established in
connection with the Borrower's employee benefits plans, the trustee of which is
Wachovia Bank, N.A.); (ii) approximately $27,600,000 of the proceeds of such
acquisition (the "Flexitrust Tender Proceeds") would be used by the Flexitrust
to repay in full the purchase money note of the Flexitrust to the Borrower which
was executed by the Flexitrust in connection with the original purchase of such
Capital Stock of the Borrower by the Flexitrust; and (iii) the Flexitrust Tender
Proceeds so paid to the Borrower in repayment of such purchase money note would
be distributed by the Borrower to IHC, either as a loan or other Investment or
as a Restricted Payment (any such type of distribution being the "Flexitrust
Tender Proceeds Distribution"). Effective upon satisfaction of the conditions to
effectiveness set forth in Section 11 hereof, the Agent and the Banks (x)
consent to the Flexitrust Tender Proceeds Distribution, as part of the
Flexitrust Tender Proceeds Actions, and (y) solely in connection with the
Flexitrust Tender Proceeds Actions, waive the provisions of Sections 6.05, 6.16,
6.17 and 6.25 of the Credit Agreement and agree that no Default or Event of
Default shall occur under the Credit Agreement by virtue of the Flexitrust
Tender Proceeds Distribution.

         3. Restatement of Representations and Warranties. The Borrower hereby
restates and renews each and every representation and warranty heretofore made
by it in the Credit Agreement and the other Loan Documents as fully as if made
on the date hereof and with specific reference to this Consent and Waiver and
all other loan documents executed and/or delivered in connection herewith.

         4. Effect of Consent and Waiver. Except as set forth expressly
hereinabove, all terms of the Credit Agreement and the other Loan Documents
shall be and remain in full force and effect, and shall constitute the legal,
valid, binding and enforceable obligations of the Borrower. The consent and
waiver set forth in Section 2 hereof shall relate only to the Flexitrust Tender
Proceeds Distributions as part of the Flexitrust Tender Proceeds Actions, and
shall be deemed to have prospective application only, unless otherwise
specifically stated herein.

         5. Ratification. The Borrower hereby restates, ratifies and reaffirms
each and every term, covenant and condition set forth in the Credit Agreement
and the other Loan Documents effective as of the date hereof.




                                       69

<PAGE>   3




         6. Counterparts. This Consent and Waiver may be executed in any number
of counterparts and by different parties hereto in separate counterparts and
transmitted by facsimile to the other parties, each of which when so executed
and delivered by facsimile shall be deemed to be an original and all of which
counterparts, taken together, shall constitute but one and the same instrument.

         7. Section References. Section titles and references used in this
Consent and Waiver shall be without substantive meaning or content of any kind
whatsoever and are not a part of the agreements among the parties hereto
evidenced hereby.

         8. No Default. To induce the Agent and the Banks to enter into this
Consent and Waiver and to continue to make advances pursuant to the Credit
Agreement, the Borrower hereby acknowledges and agrees that, as of the date
hereof, and after giving effect to the terms hereof, there exists (i) no Default
or Event of Default and (ii) no right of offset, defense, counterclaim, claim or
objection in favor of the Borrower arising out of or with respect to any of the
Loans or other obligations of the Borrower owed to the Banks under the Credit
Agreement.

         9. Further Assurances. The Borrower agrees to take such further actions
as the Agent shall reasonably request in connection herewith to evidence the
amendments herein contained to the Borrower.

         10. Governing Law. This Consent and Waiver shall be governed by and
construed and interpreted in accordance with, the laws of the State of Georgia.

         11. Conditions Precedent. This Consent and Waiver shall become
effective only upon execution and delivery of this Consent and Waiver by each of
the parties hereto.



                                      70

<PAGE>   4



         IN WITNESS WHEREOF, the Borrower, the Agent and each of the Banks has
caused this Consent and Waiver to be duly executed, under seal, by its duly
authorized officer as of the day and year first above written.


<TABLE>
<S>                                                  <C>
SAVANNAH FOODS & INDUSTRIES,                         WACHOVIA BANK, N.A. (successor by
INC., as Borrower     (SEAL)                         merger to Wachovia Bank of Georgia,
                                                     N.A.), as Agent and as a Bank  (SEAL)

By:                                                  By:
   -----------------------------------                   ----------------------------------
   Title:                                                Title:

THE CHASE MANHATTAN BANK,                            THE FIRST NATIONAL BANK OF CHICAGO,
as a Bank           (SEAL)                           as a Bank                   (SEAL)


By:                                                  By:
   -----------------------------------                   ----------------------------------
   Title:                                                Title:


CITIZENS BANK (formerly                              SUNTRUST BANK, ATLANTA,
Second National Bank                                 as a Bank                   (SEAL)
of Saginaw, as a Bank  (SEAL)

By:                                                  By:
   -----------------------------------                   ----------------------------------
   Title:                                                Title:


COOPERATIEVE CENTRALE RAIFFEISEN-
BOERENLEEENBANK B.A., "RABOBANK
NEDERLAND", NEW YORK BRANCH,
as a Bank             (SEAL)


By:
   -----------------------------------
   Title:

By:
   -----------------------------------
   Title:
</TABLE>



                                      71