<PAGE>   1

                                                                    EXHIBIT 4.9

                           SECURED  PROMISSORY  NOTE

                                  (Term Note)


$11,000,000.00
                                                                 March 20, 1998
                                                               Atlanta, Georgia


         FOR  VALUE  RECEIVED, the undersigned (hereinafter referred to
collectively as "Borrowers" and individually as a "Borrower"), hereby jointly
and severally promise to pay to the order of FLEET CAPITAL CORPORATION, a Rhode
Island corporation (hereinafter "Lender"), in such coin or currency of the
United States which shall be legal tender in payment of all debts and dues,
public and private, at the time of payment, the principal sum of $11,000,000,
together with interest from and after the date hereof on the unpaid principal
balance outstanding at a variable rate per annum as set forth in Section 2.1 of
the Loan Agreement (as such term is defined below) and computed as set forth in
Section 2.3 of the Loan Agreement.

         This Secured Promissory Note (the "Note") is the Term Note referred to
in, and is issued pursuant to, that certain Loan and Security Agreement dated
March 20, 1998, among Borrowers and Lender  (hereinafter, as amended from time
to time, the "Loan Agreement"), and is entitled to all of the benefits and
security of the Loan Agreement.  All of the terms, covenants and conditions of
the Loan Agreement and the Security Documents are hereby made a part of this
Note and are deemed incorporated herein in full.  All capitalized terms used
herein, unless otherwise specifically defined in this Note, shall have the
meanings ascribed to them in the Loan Agreement.

         The principal amount and accrued interest of this Note shall be due
and payable on the dates and in the manner hereinafter set forth:

                 (a)  Interest shall be due and payable monthly, in arrears, on
         the first day of each month, commencing on April 1, 1998, and
         continuing until such time as the full principal balance, together
         with all other amounts owing hereunder, shall have been paid in full;

                 (b)  Principal shall be due and payable monthly commencing on
         April 1, 1998, and continuing on the first day of each month
         thereafter to and including the first day of February, 2003, in
         installments in the amounts set forth on Schedule 1 attached hereto;
         and

                 (c)  The entire remaining principal amount then outstanding,
         together with any and all other amounts due hereunder, shall be due
         and payable on March 1, 2003.





<PAGE>   2


If, prior to the date on which this Note is required to be paid in full in
accordance with the foregoing provisions, the Loan Agreement is terminated
pursuant to Section 5.2 of the Loan Agreement, then the entire unpaid principal
balance and accrued interest on this Note shall be immediately due and payable
in full and shall be paid on the effective date of such termination.

         Borrowers shall prepay this Note as provided in Section 4.3.4 of the
Loan Agreement without any prepayment premium.  Borrowers may prepay this Note
in whole at any time or in part from time to time in accordance with the
provisions of Section 4.3.5 of the Loan Agreement, provided that each such
prepayment shall be made together with accrued interest on the principal amount
so prepaid at the prepayment date plus a premium at the applicable percentage
set forth below:

  If Prepayment is Made:            The Premium Shall Be:
  ---------------------             --------------------
  During the first Loan Year        2% of the principal amount paid

  During the second Loan Year       1% of the principal amount paid

  During the third Loan Year        .5% of the principal amount paid

         All partial prepayments, whether mandatory or voluntary, shall be
applied to installments of principal in the inverse order of their maturities.
Notwithstanding the foregoing, no prepayment premium shall be payable by
Borrowers to the extent such partial prepayment is made from proceeds of APB's
sale of the assets of its beverage division, provided, that such prepayment is
made not more than 3 days after the closing date of such sale.

         Upon or after the occurrence of an Event of Default, Lender shall have
all of the rights and remedies set forth in Section 11 of the Loan Agreement,
including the right to declare the then outstanding principal balance and
accrued interest hereof to be and the same shall thereupon become, immediately
due and payable without notice to or demand upon any Borrower, all of which
each Borrower hereby expressly waives.

         Time is of the essence of this Note.  To the fullest extent permitted
by Applicable Law, each Borrower, for itself and its legal representatives,
successors and assigns, expressly waives presentment, demand, protest, notice
of dishonor, notice of non-payment, notice of maturity, notice of protest,
presentment for the purpose of accelerating maturity, diligence in collection,
and the benefit of any exemption or insolvency laws.

         Wherever possible, each provision of this Note shall be interpreted in
such manner as to be effective and valid under Applicable Law, but if any
provision of this Note shall be prohibited or invalid under Applicable Law,
such provision shall be ineffective to the extent of such prohibition or
invalidity without invalidating the remainder of such provision or remaining
provisions of this Note.  No delay or failure on the part of Lender in the
exercise of any right or remedy hereunder shall operate as a waiver thereof,
nor as an acquiescence in any default,





                                    - 2 -
<PAGE>   3

nor shall any single or partial exercise by Lender of any right or remedy
preclude any other right or remedy.  Lender, at its option, may enforce its
rights against any collateral securing this Note without enforcing its rights
against any Borrower, any guarantor of the indebtedness evidenced hereby or any
other property or indebtedness due or to become due to any Borrower.  Each
Borrower agrees that, without releasing or impairing such Borrower's liability
hereunder, Lender may at any time release, surrender, substitute or exchange
any collateral securing this Note and may at any time release any party
primarily or secondarily liable for the indebtedness evidenced by this Note.

         This Note shall be governed by, and construed and enforced in
accordance with, the laws of the State of Georgia and is intended to take
effect as an instrument under seal.





                                    - 3 -
<PAGE>   4

         IN  WITNESS  WHEREOF,  each Borrower has caused this Note to be duly
executed and delivered in Atlanta, Georgia, on the date first above written.


                                  ATLANTIC PREMIUM BRANDS, LTD.,
ATTEST:                           a Delaware corporation ("Borrower")


/s/ TOM D. WIPPMAN                By:  /s/ MERRICK M. ELFMAN
-------------------------              ---------------------
Secretary                              Merrick M. Elfman, Chairman
[CORPORATE SEAL]

                                  CARLTON FOODS CORP.,
ATTEST:                           a Delaware corporation ("Borrower")


/s/ TOM D. WIPPMAN                By:  /s/ MERRICK M. ELFMAN
-------------------------              ---------------------
Secretary                              Merrick M. Elfman, Chairman
[CORPORATE SEAL]

                                  PREFCO CORP.,
ATTEST:                           a Delaware corporation ("Borrower")


/s/ TOM D. WIPPMAN                By:  /s/ MERRICK M. ELFMAN
-------------------------              ---------------------
Secretary                              Merrick M. Elfman, Chairman
[CORPORATE SEAL]


                  [Signatures continued on the following page]





                                    - 4 -
<PAGE>   5

                                  GROGAN'S FARM, INC.,
ATTEST:                           a Delaware corporation ("Borrower")


/s/ TOM D. WIPPMAN                By:  /s/ MERRICK M. ELFMAN
-------------------------              ---------------------
Secretary                              Merrick M. Elfman, Chairman
[CORPORATE SEAL]


                                  RICHARDS CAJUN FOODS CORP.,
ATTEST:                           a Delaware corporation ("Borrower")


/s/ TOM D. WIPPMAN                By:  /s/ MERRICK M. ELFMAN
-------------------------              ---------------------
Secretary                              Merrick M. Elfman, Chairman
[CORPORATE SEAL]


                                  POTTER'S ACQUISITION CORP.,
ATTEST:                           a Delaware corporation ("Borrower")


/s/ TOM D. WIPPMAN                By:  /s/ MERRICK M. ELFMAN
-------------------------              ---------------------
Secretary                              Merrick M. Elfman, Chairman
[CORPORATE SEAL]





                                    - 5 -
<PAGE>   6


                                   SCHEDULE 1




<TABLE>
<CAPTION>
                 Period                            Installment Amount
                 ------                            ------------------
         <S>                                       <C>
         April 1, 1998 through                     $76,400.00
          March 1, 1999, inclusive

         April 1, 1999 through                     $114,500.00
          March 1, 2000, inclusive

         April 1, 2000 through                     $152,800.00
          February 1, 2001, inclusive

         April 1, 2001 through                     $191,000.00
          March 1, 2003, inclusive
</TABLE>





                                    - 6 -