<PAGE>
                                 EXHIBIT 10.2.1

                    SECOND AMENDMENT TO AMENDED AND RESTATED
                          LOAN AND SECURITY AGREEMENT

         This SECOND AMENDMENT TO AMENDED AND RESTATED LOAN AND SECURITY
AGREEMENT (this "Amendment") is entered into as of November ___, 2004, among
DELTA APPAREL, INC., a Georgia corporation ("Borrower"), Lenders signatory
hereto, and CONGRESS FINANCIAL CORPORATION (SOUTHERN), as Agent ("Agent").

                                  WITNESSETH:

         WHEREAS, Borrower, Agent, Guarantors and the Lenders party thereto
from time to time are parties to that certain Amended and Restated Loan and
Security Agreement dated as of October 3, 2003, as amended by that certain
First Amendment to Amended and Restated Loan and Security Agreement dated as of
August 30, 2004 (as amended, restated, refinanced, supplemented or otherwise
modified from time to time, the "Loan Agreement"; capitalized terms used herein
and not otherwise defined shall have the meanings ascribed to such terms in the
Loan Agreement), pursuant to which Agent and Lenders have agreed to make
Advances, issue or arrange for the issuance of Letters of Credit and make other
extensions of credit to Borrower from time to time pursuant to the terms and
conditions thereof and the other Financing Agreements;

         WHEREAS, Borrower has requested that certain terms and conditions of
the Loan Agreement be amended; and

         WHEREAS, Agent, Lenders and, by their respective acknowledgment
hereof, Guarantors have agreed to the requested amendments on the terms and
conditions provided herein;

         NOW THEREFORE, in consideration of the foregoing premises and other
good and valuable consideration, the receipt and sufficiency of which are
hereby acknowledged, the parties hereto hereby agree as follows:

         1.       Amendments to the Loan Agreement.

         (a)      Section 1 of the Loan Agreement, Definitions, is hereby
                  amended and modified by amending and restating the
                  definitions of "Borrowing Base", "Commitment", "Maximum
                  Credit" and "Mortgages", in their respective entirety as
                  follows:

         ""Borrowing Base" shall mean, at any time, the amount equal to:

         (a)      the sum of:

                  (i)      eighty-five (85%) percent of the Net Amount of the
                           Eligible Accounts, plus

                  (ii)     the lesser of:

                           (1)      the Inventory Loan Limit, or

                           (2)      fifty-five (55%) percent of the Value of
                                    Eligible Inventory consisting of finished
                                    goods, raw materials consisting of raw
                                    cotton and yarn for such finished goods,
                                    and finished yarn categorized as
                                    work-in-process; plus

                  (iii)    to the extent greater than zero, the lesser of:

                           (1)      (A)      the Fixed Asset Loan Limit, minus

                                    (B)      the Fixed Asset Loan Amortization
                                             Amount, or


                                       1
<PAGE>
                           (2)      (A)     eighty-five percent (85%) of the
                                            appraised Net Orderly Liquidation
                                            Value of Eligible Equipment (other
                                            than the Tennessee Equipment)
                                            determined from time to time by a
                                            qualified appraiser acceptable to
                                            Agent, minus

                                    (B)      the Fixed Asset Loan Amortization
                                             Amount, plus

                  (iii)    the result of:

                           (1)      the Tennessee Asset Loan Limit, minus

                           (2)      the Tennessee Asset Loan Amortization
                                    Amount, minus

         (b)      Reserves.

         "Commitment" shall mean, at any time, as to each Lender, the principal
amount set forth beside such Lender's name on Schedule 1.21 hereto or in the
Assignment and Acceptance pursuant to which such Lender became a Lender
hereunder in accordance with the provisions of Section 13.6 hereof, as the same
may be adjusted from time to time in accordance with the terms hereof;
sometimes being collectively referred to herein as "Commitments".

         "Maximum Credit" shall mean the amount of $42,750,000.

         "Mortgages" shall mean, individually and collectively, each of the
following, as each may be amended, modified, supplemented, extended or restated
from time to time: (a) that certain Amended and Restated Deed of Trust,
Assignment of Rents and Leases, Security Agreement and Fixture Filing, dated
October 3, 2003, by Borrower in favor of Agent with respect to the Real
Property and related assets of Borrower located in Catawba County, North
Carolina; (b) that certain Amended and Restated Deed of Trust, Assignment of
Rents and Leases, Security Agreement and Fixture Filing, dated October 3, 2003,
by Borrower in favor of Agent with respect to the Real Property and related
assets of Borrower located in Knox County, Tennessee; (c) that certain Amended
and Restated Mortgage, Assignment of Rents and Leases and Security Agreement,
dated October 3, 2003, by Borrower in favor of Agent with respect to the Real
Property and related assets of Borrower located in Edgefield County, South
Carolina; (d) that certain Mortgage, Assignment of Rents and Leases and
Security Agreement dated as of October 3, 2003, with respect to the Real
Property and related assets of Borrower located in Fayette County, Alabama; and
(e) that certain Deed of Trust, Assignment of Rents and Leases, Security
Agreement and Fixture Filing, dated November 1, 2004, by Borrower in favor of
Agent with respect to the Real Property and related assets of Borrower located
in Anderson County, Tennessee."

         (b)      Section 1 of the Loan Agreement, Definitions, is hereby
                  further amended and modified by adding the following to the
                  end of the first sentence of the definition of "Reserves":

                  "; or (g) $800,000 in respect of the Tennessee Equipment to
                  be acquired by Borrower, provided that such reserve shall be
                  reduced from time to time in the amount of eighty percent
                  (80%) of Borrower's invoice cost (net of shipping, freight,
                  installation, and other so-called "soft costs") of new
                  Tennessee Equipment that is purchased by Borrower upon
                  receipt by Agent of (i) a copy of the invoice relating to
                  such Tennessee Equipment, (ii) a description of such
                  Tennessee Equipment, including, without limitation, the
                  manufacturer and vendor of such Tennessee Equipment and, if
                  applicable, the make, model and serial number of such
                  Tennessee Equipment, and (iii) evidence, in form and
                  substance satisfactory to Agent, of the receipt and the
                  installation of such Tennessee Equipment"

         (c)      Section 1 of the Loan Agreement, Definitions, is hereby
                  further amended and modified by adding the following
                  definitions in the appropriate alphabetical order:


                                       2
<PAGE>

         ""Tennessee Asset Loan Amortization Amount" shall mean (a) until the
earlier of the Renewal Date (as it may be extended pursuant to Section 13.1 of
this Agreement) and October 1, 2009, an amount equal to (i) from and after
December 1, 2004, through December 30, 2004, $15,278, plus (ii) from and after
the first day of each month commencing January 1, 2005, and through the last
day of each such month, the product of: (1) $15,278 multiplied by (2) the
cumulative number of months that have elapsed since December 1, 2004 and (b)
after the earlier of the Renewal Date (as it may be extended pursuant to
Section 13.1 of this Agreement) and October 1, 2009, $2,750,000.

         "Tennessee Asset Loan Limit" shall mean $2,750,000.

         "Tennessee Equipment" shall mean Eligible Equipment to be purchased by
Borrower for installation and use at the facility located on Real Property
owned by Borrower in Anderson County, Tennessee, and on which Agent has a
Mortgage, including, without limitation, Eligible Equipment in the form of
racking and forklifts."

         (d)      Section 13.3 of the Loan Agreement, Notices, is hereby
                  amended and modified by adding the following Section 13.3 to
                  the end of such Section:

                  "13.3 Notices. All notices, requests and demands hereunder
         shall be in writing and deemed to have been given or made: if
         delivered in person, immediately upon delivery; if by telex, telegram
         or facsimile transmission, immediately upon sending and upon
         confirmation of receipt; if by nationally recognized overnight courier
         service with instructions to deliver the next Business Day, one (1)
         Business Day after sending; and if by certified mail, return receipt
         requested, five (5) days after mailing. All notices, requests and
         demands upon the parties are to be given to the following addresses
         (or to such other address as any party may designate by notice in
         accordance with this Section):

                  If to Borrower:       DELTA APPAREL, INC.
                                        2750 Premiere Parkway, Suite 100
                                        Duluth, GA 30097
                                        Attention: Herb Mueller
                                        Telephone No.: (678) 775-6900
                                        Telecopy No.: (678) 584-1880

                  If to Agent:          CONGRESS FINANCIAL CORPORATION
                                        (SOUTHERN)
                                        110 East Broward Blvd.
                                        Suite 2050
                                        Fort Lauderdale, Florida 33301
                                        Attention: Kerry Maxwell, Portfolio
                                                   Manager
                                        Telephone No.: (954) 467-2262
                                        Telecopy No.: (954) 467-5520"

         (e)      The Loan Agreement is hereby amended and modified by adding
                  the Schedule 1.21 attached hereto as Schedule 1.21 to the
                  Loan Agreement.

         2.       Commitments; Pro Rata Shares. Each of the Lenders
acknowledges and agrees that, as of the effectiveness of this Amendment, (a)
the Commitment of such Lender is as set forth below such Lender's signature on
the signature pages to this Amendment and (b) the Pro Rata Share of such Lender
is as set forth below such Lender's signature on the signature pages to this
Amendment.

         3.       No Other Amendments or Waivers. Except in connection with the
amendments set forth above, the execution, delivery and effectiveness of this
Amendment shall not operate as a waiver of any right, power or remedy of Agent
or Lenders under the Loan Agreement or any of the other Financing Agreements,
nor constitute a waiver of any provision of the Loan Agreement or any of the
other Financing Agreements. Except for the amendments set forth above, the text
of the Loan Agreement and all other Financing Agreements shall remain unchanged
and in full force and effect and Borrower hereby ratifies and confirms its
obligations thereunder. This


                                       3
<PAGE>
Amendment shall not constitute a modification of the Loan Agreement or any of
the other Financing Agreements or a course of dealing with Agent or Lenders at
variance with the Loan Agreement or the other Financing Agreements such as to
require further notice by Agent or Lenders to require strict compliance with
the terms of the Loan Agreement and the other Financing Agreements in the
future, except as expressly set forth herein. Borrower acknowledges and
expressly agrees that Agent and Lenders reserve the right to, and do in fact,
require strict compliance with all terms and provisions of the Loan Agreement
and the other Financing Agreements, as amended herein. Borrower has no
knowledge of any challenge to Agent's or any Lender's claims arising under the
Financing Agreements, or to the effectiveness of the Financing Agreements.

         4.       Conditions Precedent to Effectiveness. This Amendment shall
become effective when, and only when, Agent shall have received:

         (a)      counterparts of this Amendment duly executed and delivered by
                  Borrower, Agent and the Lenders;

         (b)      with respect to each Mortgage, an opinion from local counsel
                  with respect to this Amendment or a title certification or
                  title opinion stating that no security interest, mortgage,
                  pledge, lien, charge or other encumbrance of any nature shall
                  have been recorded since the filing of such Mortgage;

         (c)      payment, for the benefit of the Lenders according to their
                  Pro Rata Shares of the Commitment, of a non-refundable fee in
                  the amount of $13,750, due and payable on the date hereof;
                  and

         (d)      such other information, documents, instruments or approvals
                  as Agent or Agent's counsel may reasonably require.

         5.       Representations and Warranties of Borrower. In consideration
of the execution and delivery of this Amendment by Agent and Lenders, Borrower
hereby represents and warrants in favor of Agent and Lenders as follows:

         (a)      the execution, delivery and performance of this Amendment and
                  the transactions contemplated hereunder are (i) all within
                  Borrower's corporate powers, (ii) have been duly authorized,
                  (iii) are not in contravention of law or the terms of
                  Borrower's certificate of incorporation, by-laws, or other
                  organizational documentation, or any indenture, agreement or
                  undertaking to which Borrower is a party or by which Borrower
                  or its property are bound and (iv) do not result in or
                  require the creation or imposition of any Lien upon or with
                  respect to any of the properties of Borrower or any of its
                  Subsidiaries;

         (b)      this Amendment has been duly authorized, validly executed and
                  delivered by one or more authorized signatories of Borrower
                  and constitutes the legal, valid and binding obligation of
                  Borrower, enforceable against Borrower in accordance with its
                  terms;

         (c)      the execution, delivery and performance of this Amendment
                  does not and will not require the consent or approval of any
                  regulatory authority or governmental authority or agency
                  having jurisdiction over Borrower that has not already been
                  obtained;

         (d)      no Default or Event of Default exists under the Loan
                  Agreement or the other Financing Agreements;

         (e)      as of the date hereof and after giving effect to this
                  Amendment, all representations and warranties of Borrower and
                  Guarantors set forth in the Loan Agreement and the other
                  Financing Agreements are true, correct and complete in all
                  material respects; and


                                       4
<PAGE>
         (f)      all Financing Agreements to which Borrower is a party,
                  including, without limitation, the Loan Agreement, constitute
                  valid and legally binding obligations of Borrower and are
                  enforceable against Borrower in accordance with the terms
                  thereof.

         6.       Counterparts. This Amendment may be executed in multiple
counterparts, each of which shall be deemed to be an original and all of which,
taken together, shall constitute one and the same agreement. In proving this
Amendment in any judicial proceedings, it shall not be necessary to produce or
account for more than one such counterpart signed by the party against whom
such enforcement is sought. Any signatures delivered by a party by facsimile
transmission or by e-mail transmission of an adobe file format document (also
known as a PDF file) shall be deemed an original signature hereto.

         7.       Reference to and Effect on the Financing Agreements. Upon the
effectiveness of this Amendment, on and after the date hereof, each reference
in the Loan Agreement to "this Agreement," "hereunder," "hereof" or words of
like import referring to the Loan Agreement, and each reference in the other
Financing Agreements to "the Loan Agreement" "thereunder," "thereof" or words
of like import referring to the Loan Agreement, shall mean and be a reference
to the Loan Agreement as amended hereby.

         8.       Affirmation of Guaranty. By executing this Amendment, each
Guarantor hereby acknowledges, consents and agrees that all of its obligations
and liability under the Guarantee to which it is a party and the other
Financing Documents to which it is a party remain in full force and effect, and
that the execution and delivery of this Amendment and any and all documents
executed in connection herewith shall not alter, amend, reduce or modify its
obligations and liability under such Guarantee or any of the other Financing
Agreements to which it is a party.

         9.       Costs, Expenses and Taxes. Borrower agrees to pay on demand
all costs and expenses in connection with the preparation, execution, and
delivery of this Amendment and the other instruments and documents to be
delivered hereunder, including, without limitation, the fees and out-of-pocket
expenses of counsel for Agent with respect thereto and with respect to advising
Agent as to its rights and responsibilities hereunder and thereunder. In
addition, Borrower agrees to pay any and all stamp and other taxes payable or
determined to be payable in connection with the execution and delivery of this
Amendment and the other instruments and documents to be delivered hereunder,
and agree to save Agent and Lenders harmless from and against any and all
liabilities with respect to or resulting from any delay in paying or omission
to pay such taxes. Borrower hereby acknowledges and agrees that Agent may,
without prior notice to Borrower, charge such costs and fees to Borrower's loan
account pursuant to the Loan Agreement, which amounts shall constitute Loans
under the Loan Agreement.

         10.      Section Titles. The section titles contained in this
Amendment are included for the sake of convenience only, shall be without
substantive meaning or content of any kind whatsoever, and are not a part of
the agreement between the parties.

         11.      Entire Agreement. This Amendment and the other Financing
Agreements constitute the entire agreement and understanding between the
parties hereto with respect to the transactions contemplated hereby and
supersede all prior negotiations, understandings and agreements between such
parties with respect to such transactions.

         12.      GOVERNING LAW. THE VALIDITY, INTERPRETATION AND ENFORCEMENT
OF THIS AMENDMENT SHALL BE GOVERNED BY THE LAWS OF THE STATE OF GEORGIA.

         13.      Financing Agreement. This Amendment shall be deemed to be a
Financing Agreement for all purposes.


                 [Remainder of page intentionally left blank.]



                                       5
<PAGE>
         IN WITNESS WHEREOF, the parties hereto have executed and delivered
this Amendment as of the day and year first written above.


BORROWER:                       DELTA APPAREL, INC., a Georgia corporation


                                By:
                                   --------------------------------------------
                                Title:
                                      -----------------------------------------

AGENT AND
LENDERS:                        CONGRESS FINANCIAL CORPORATION (SOUTHERN), as
                                Agent and a Lender

                                By:
                                   --------------------------------------------
                                Title:
                                      -----------------------------------------

                                COMMITMENT:  $25,756,875
                                PRO RATA SHARE: 60.25%


                                ING CAPITAL LLC, as a Lender


                                By:
                                   --------------------------------------------
                                Title:
                                      -----------------------------------------
                                COMMITMENT: $8,550,000
                                PRO RATA SHARE: 20.0%


                                SIEMENS FINANCIAL SERVICES, INC., as a Lender

                                By:
                                   --------------------------------------------
                                Title:
                                      -----------------------------------------
                                COMMITMENT: $8,443,125

                                PRO RATA SHARE: 19.75%


ACKNOWLEDGED AND
AGREED:
GUARANTORS:                     M.J. SOFFE CO., a North Carolina corporation

                                By:
                                   --------------------------------------------
                                Title:
                                      -----------------------------------------


                                SAIM, LLC, a North Carolina limited liability
                                company

                                By:
                                   --------------------------------------------
                                Title:
                                      -----------------------------------------


                                       6
<PAGE>
                                 Schedule 1.21
                                  Commitments


<TABLE>
<CAPTION>
               LENDER                                             COMMITMENT

<S>                                                               <C>
Congress Financial Corporation (Southern)                        $25,756,875
ING Capital LLC                                                   $8,550,000
Siemens Financial Services, Inc.                                  $8,443,125


All Lenders                                                      $42,750,000
</TABLE>


                                       7