<PAGE>

                                                                    Exhibit 10.4


                       FIRST AMENDMENT TO AGENCY AGREEMENT
                       -----------------------------------

          THIS FIRST AMENDMENT TO AGENCY AGREEMENT (the "Amendment"), is made
and entered into effective as of the 3rd day of May, 2001 (the "Effective
Date"), by and between LONATI S.P.A., a corporation organized under the laws of
the Italy with an office in Brescia, Via S. Polo 11, Brescia, Italy and a legal
residence at Via Osculati 9. Milano, Italy ("Lonati"), successor to Lonati
S.r.L., a corporation organized under the laws of the Italy ("Lonati S.r.L.")
and SPEIZMAN INDUSTRIES, INC., a business entity organized under the laws of
Delaware, having offices at 701 Griffith Road, Charlotte, North Carolina, United
States of America ("Speizman").

                             PRELIMINARY STATEMENT:

          Lonati is engaged in the manufacture, marketing, distribution and sale
of single and double cylinder sock knitting machines and spare parts described
in the attached Exhibit A (together with all other goods, machines and spare
parts sold by Lonati to Speizman from time to time, the "Products"); and

          Speizman and Lonati, S.r.L. are parties to that certain Agency
Agreement dated January 2, 1992 and Annex No. 1 attached thereto (the "Original
Agreement");

          Speizman and Lonati wish to enter this Amendment for the purpose of
confirming and agreeing that Lonati has succeeded to the rights and obligations
of Lonati, S.r.L. under the Original Agreement and for the purpose of amending
the Original Agreement as set forth herein;

                                   AGREEMENT:

          NOW THEREFORE, in consideration of the premises and the mutual
promises and covenants set forth herein, the parties hereby agree as follows:

     1.   Succession by Lonati to Rights and Obligations of Lonati, S.r.L.
          ----------------------------------------------------------------
Lonati and Speizman acknowledge and agree that Lonati has succeeded to all
rights and obligations of Lonati, S.r.L. under the Original Agreement as amended
hereby. All references in the Original Agreement to "Lonati" shall mean and
refer to Lonati, S.p.A.

     2.   Term. Paragraph 2 of the Original Agreement entitled "Term" is hereby
          ----
amended by deleting Paragraph 2 in its entirety and substituting the following
therefor:

________________________

** Approximately three pages of confidential material have been omitted pursuant
to a request for confidential treatment. The omitted material has been filed
separately with the Securities and Exchange Commission.

                                        1

<PAGE>

     "2.  TERM. The parties acknowledge that the original term of this Agreement
          ----
     began January 2, 1992 and continued for an initial term through December
     31, 1995. Thereafter, the term of this Agreement has been continued from
     year to year for one year terms beginning on January 1 of each calendar
     year and expiring on December 31 of each calendar year. The term of the
     Agreement is hereby extended effective as of May 3, 2001 for a term, unless
     sooner terminated as hereinafter provided, continuing through December 31,
     2003 (the "Extended Term"). Thereafter, the Agreement will automatically be
     renewed for additional one (1) year successive terms (each a "Renewal
     Term"). Either party may terminate this Agreement at the end of the
     Extended Term or any Renewal Term by giving the other party at lease ninety
     (90) days prior written notice."

     3.   Terms of Payment. Subparagraph D of Paragraph 6 of the Original
          ----------------
Agreement entitled "Term" is hereby amended by deleting Subparagraph D of
Paragraph 6 in its entirety and substituting the following therefor:

          "D.  TERMS OF SALE AND PAYMENT
               -------------------------

               (1) Lonati shall sell the products to Speizman at such prices and
     on such terms and conditions as shall be established by Lonati and in
     effect on the date Speizman's order is accepted. Prices shall be subject to
     change by Lonati at any time and without notice. All prices quoted by
     Lonati to Speizman shall be a net price EXW (Ex Works) Lonati's
     manufacturing facility in Brescia, Italy (per INCOTERMS 1990) and crated
     for sea shipment, unless otherwise designated by Lonati. Special packing or
     handling, other than in accordance with Lonati's standard commercial export
     practices in effect at the time of shipment, shall be at the sole expense
     of Speizman.

               (2) The terms of payment for Products sold by Lonati to Speizman
     pursuant to purchase orders accepted during the period beginning on the
     Effective Date and ending on December 31, 2002 (the "Revolving Credit
     Period") shall be as follow:

               (a) 10% of the purchase price for products covered by any
          purchase order from Speizman which has been accepted by Lonati shall
          be paid by wire transfer or other immediate funds with the purchase
          order and prior to shipment;

               (b) 50% of the purchase price shall be paid by irrevocable
          letter of credit issued at least 7 days prior to the shipment date,
          payable at 60 (sixty) days from the date of issuance of the Bill of
          Lading for the shipment of the Products and meeting the other
          requirements for letters of credit set forth below;

               (c) The remaining 50% of the purchase price shall be paid
          within 150 days of shipment (or, if earlier, upon the expiration of
          the Revolving Credit Period on December 31, 2002) by wire transfer
          upon an invoice from Lonati, provided, however, that the unpaid
          balance owed by Speizman to Lonati with respect to the purchase price
          for all Products (and related services) purchased by Speizman
          (excluding accrued and unpaid interest payable on such unpaid balance)
          shall at no

                                        2

<PAGE>

          time exceed Eight Billion Three Hundred Twenty-Three Million It. Lira
          (8,323,000,000 ITL) (4,298,470.771 EUR).

               The portion of the purchase price which is payable after shipment
          pursuant to subparagraph 2(c) above and which is not payable pursuant
          to an irrevocable letter of credit as provided in subparagraph 2(b)
          above is sometime referred to herein as the "Financed Portion of the
          Purchase Price." It is expressly agreed that the term "Financed
          Portion of the Purchase Price" shall include all amounts payable
          pursuant to subparagraph 2(c) above after shipment, including without
          limitation the unpaid 40% portion of the purchase price for Products
          which have been shipped but with respect to which the payment is not
          yet due (i.e. with respect to which the period of 150 days after
          shipment has not yet elapsed) and the unpaid 40% portion of the
          purchase price for Products which have been shipped and with respect
          to which the balance of the purchase price is due and payable.

               The unpaid balance of the purchase price shall bear interest
          beginning 90 days after shipment of the purchased Product(s) and
          continuing until paid in full at an annual rate of 7%. Interest shall
          be payable monthly in arrears on the 1st day of each month until he
          outstanding balance of the purchase price is paid in full. Speizman's
          obligation to pay the balance of the purchase price shall be secured
          by a second priority security interest and lien on all of Speizman's
          inventory, accounts and any and all rights to the payment of money or
          other forms of consideration of any kind for goods sold or leased or
          for services rendered. Concurrently with the execution of the First
          Amendment to Agency Agreement amending this Agreement, Speizman has
          executed and delivered a security agreement granting Lonati a security
          interest in such collateral and has executed and delivered UCC
          Financing Statements. Speizman covenants, represents and warrants that
          such security interest is, and shall remain during the terms of the
          Agreement, second in priority only to a first priority security
          interest in such property granted by Speizman to SouthTrust Bank, N.A.
          to secure a line of credit and other financing in the aggregate
          existing principal amount of U.S. $27,500,000. Speizman further
          represents and warrants that it has obtained the consent of SouthTrust
          Bank, N.A. to the second priority security interest granted to Lonati
          and has provided Lonati with written evidence of such consent.

               Except for and excluding amounts owed with respect to the
          Excluded Products, in no event shall the total outstanding balance
          (excluding accrued and unpaid interest) owed by Speizman to Lonati
          with respect to the unpaid balance of the purchase price for Products
          (and related services) purchased by Speizman, including the Financed
          Portion of the Purchase Price for all Products, exceed Eight Billion
          Three Hundred Twenty-Three Million It. Lira (8,323,000,000 ITL)
          4,298,470.771 EUR) at any time, including, without limitation all
          Products (and related services) purchased pursuant to the terms of
          Subparagraph D of the Paragraph 6. In the event that, notwithstanding
          the foregoing, the outstanding unpaid balance of the purchase prior
          for Products (and related services) purchased by Speizman (excluding
          accrued and unpaid interest) exceeds Eight Billion Three

                                        3

<PAGE>

          Hundred Twenty-Three Million It. Lira (8,323,000,000 ITL)
          (4,298,470.771 EUR) at any time and for any reason (including, without
          limitation, as a result of the purchase of Products, the failure by
          Speizman to pay amounts owed when due, changes in the exchange rate of
          US Dollars to It. Lira (EUROS), or any other reason whatsoever),
          Speizman shall immediately pay by wire transfer such funds as are
          required to reduce the amount outstanding. The term "Excluded
          Products" shall mean (i) trial machines sold by Lonati on approval to
          Speizman from time to time pursuant to Paragraph 7 of this Agreement
          and (ii) spare parts, which shall be sold on terms established by
          Lonati from time to time.

               If at any time the shipment of any Product(s) would cause the
          outstanding balance of all amounts owed by Speizman with respect to
          the purchase of Products (and related services) to exceed Eight
          Billion Three Hundred Twenty-Three Million It. Lira (8,323,000,000
          ITL) (4,298,470.771 EUR) (including the Financed Portion of the
          Purchase Price which would become owing for the Products to be
          shipped) or if such limit is exceeded for any other reason (including,
          without limitation, as a result of the purchase of Products, the
          failure by Speizman to pay amounts owed when due, changes in the
          exchange rate of US Dollars to It. Lira (EUROS), or any other reason
          whatsoever), Lonati shall have the right to suspend all shipments of
          Products (including Products for which a purchase order has been
          accepted) except upon and until (i) payment of the purchase price in
          full in advance or (ii) receipt of an irrevocable letter of credit for
          the full amount of the purchase price issued at least 7 days prior to
          the shipment date, payable at 60 (sixty) days from the Bill of Lading
          for the shipment of the Products and meeting the other requirements
          for letters of credit set forth in this Agreement or (iii) payment by
          Speizman by wire transfer of such funds as are required to reduce the
          outstanding balance of all amounts owed by Speizman with respect to
          the purchase of Products plus the Financed Portion of the Purchase
          Price of the Products to be shipped to an amount (excluding accrued
          and unpaid interest) which does not exceed Eight Billion Three Hundred
          Twenty-Three Million It. Lira (8,323,000,000 ITL) (4,298,470.771 EUR).

               The parties acknowledge that on January 1, 1999, Italy adopted
          the new Euro (EUR) currency. All references to "It. Lira" or "Lira" in
          this Agreement shall mean and include an equivalent number of Euros as
          converted from Italian Lira (ITL) based on the official fixed Euro
          rates adopted for Italy with respect to conversions for It. Lira to
          Euros. For purposes of calculating amounts owed by Speizman with
          reference tot he maximum credit limit of Eight Billion Three Hundred
          Twenty-Three Million It. Lira (8,323,000,000 ITL) (4,298,470.771 EUR),
          all amounts owed by Speizman on purchases in which the purchase price
          of the Products has been denominated on the invoice or otherwise in
          U.S. Dollars shall be converted to It. Lira and Euros using the
          exchange rates for commercial customers converting US Dollars to It.
          Lira and Euros at Banca Commerciale Italiana-Brescia, the primary bank
          used by Lonati in Italy.

                                        4

<PAGE>

                    (3) Unless the parties otherwise agree in writing, the terms
          of payment for Products sold by Lonati to Speizman pursuant to
          purchase orders accepted during the period after December 31, 2002
          during the balance of the Extended Term occurring after December 31,
          2002 or during any Renewal Term shall be as follows:

                    (a) 10% of the purchase price for Products covered by any
          purchase order from Speizman which has been accepted by Lonati during
          the period after December 31, 2002 during the balance of the Extended
          Term occurring after December 31, 2002 or during any Renewal Term
          shall be paid by wire transfer or other immediate funds with the
          purchase order and prior to shipment;

                    (b) 90% of the purchase price shall be paid by irrevocable
          letter of credit issued at least 7 days prior to the shipment date,
          payable at 60 (sixty) days from the date of issuance of the Bill of
          Lading for the shipment of the Products and meeting the other
          requirements for letters of credit set forth below;

                    (4) Except as otherwise agreed by the parties in writing
          from time to time, all purchases of Products by Speizman during the
          period beginning May 2, 2001 and ending on April 30, 2002 shall be
          paid for in United States Dollars and all letters of credit issued
          with respect to such purchases shall be payable in United States
          Dollars. With respect to purchases after April 30, 2002, Speizman
          shall pay for Products purchased after April 30, 2002 in Euros or
          United States Dollars as designated by Lonati with respect to each
          purchase and all letters of credit issued with respect to such
          purchases shall be payable in United States Dollars or Euros as
          designated by Lonati. All letters of credit must be irrevocable when
          issued and must be issued and confirmed by a bank satisfactory to
          Lonati. Other terms of the letters of credit must be satisfactory to
          Lonati."

          4.   Additional Paragraphs to Original Agreement. The following new
paragraphs are added as new Paragraphs 15, 16 and 17 to the Original Agreement:

               "15. Reimbursement of Expenses. Lonati agrees to reimburse
                    -------------------------
          Speizman for the costs of all reasonable expenses and salaries,
          including reasonable salaries, living and travel expenses for
          Speizman's technicians, as well as expenses incurred by Speizman on
          behalf of Lonati's technicians, with respect to service performed in
          up-dating the older version of the closed toe machine sold by Lonati.
          Lonati agrees to reimburse Speizman by allowing Speizman credit
          against monthly parts drafts from Lonati to Speizman. The term
          "reasonable" as used with respect to expenses, salaries and travel
          expenses shall mean such expenses, salaries and travel expenses as are
          reasonable necessary in order for Speizman to perform the required
          services in a timely manner.

                                        5

<PAGE>

16. **







































----------
** Approximately three pages of confidential material have been omitted pursuant
to a request for confidential treatment. The omitted material has been filed
separately with the Securities and Exchange Commission.

                                        6

<PAGE>

          17.  ACCEPTANCE AND LIMITED WARRANTY, DISCLAIMER OF WARRANTIES;
               ----------------------------------------------------------
               LIMITATION OF LIABILITY.
               -----------------------

               (a)  Lonati warrants to Speizman only that all Products shall be
          free from defects in mechanical parts and workmanship for a period of
          twelve (12) months from the date of delivery and shall be free of
          defects in electrical parts for a period of six (6) months from the
          date of delivery. Parts subject to normal wear and tear are not
          warranted. Speizman is not authorized to make or give additional
          warranties, expressed or implied, on behalf of Lonati.

               (b)  LONATI MAKES NO WARRANTIES OF MERCHANTABILITY OR FITNESS FOR
          ANY PARTICULAR PURPOSE, OR ANY OTHER WARRANTY, EXPRESSED OR IMPLIED,
          EXCEPT AS EXPRESSLY PROVIDED HEREIN.

               (c)  LONATI SHALL IN NO EVENT BE LIABLE, WHETHER IN CONTRACT,
          TORT, OR ON ANY OTHER BASIS, FOR CONSEQUENTIAL, INCIDENTAL, INDIRECT
          OR PUNITIVE DAMAGES, OR LOSS OF PROFITS OF ANY KIND SUSTAINED BY
          SPEIZMAN, OR BY ANY PERSON DEALING WITH SPEIZMAN, IN CONNECTION WITH
          THE PRODUCTS. LONATI'S LIABILITY FOR ANY CLAIM OF ANY KIND (INCLUDING,
          WITHOUT LIMITATION, CLAIMS BASED UPON ANY EXPRESS WARRANT CONTAINED
          HEREIN AND CLAIMS BASED UPON ANY WARRANTY IMPLIED BY LAW), SHALL BE
          LIMITED, AT LONAT'S OPTION, TO EITHER REPLACEMENT OF THE PRODUCTS OR
          THE RETURN TO SPEIZMAN OF THE PRICE PAID, AND SPEIZMAN EXPRESSLY
          WAIVES ANY RIGHT IT MIGHT HAVE TO ANY OTHER MEASURE OF DAMAGES,
          STATUTORY OR OTHERWISE.

                                        7

<PAGE>

          (d)  ALL WARRANTY DISCLAIMERS CONTINED HEREIN ARE INTENDED TO COMPLY
     WITH APPLICABLE LAW AND SHALL BE ENFORCED TO THE FULLEST EXTENT POSSIBLE
     UNDER APPLICABLE LAW. TO THE EXTENT THAT ANY WARRANTY DISCLAIMER IS DEEMED
     INVALID UNDER ANY LAW WHICH MAY BE APPLIED, ANY RELATED NON-DISCLAIMABLE
     WARRANTIES, WHETHER EXPRESS OR IMPLIED, SHALL BE LIMITED IN DURATION TO TA
     PERIOD OF SIX MONTHS.

          (e)  THE WARRANTIES CONTINED HEREIN SHALL NOT BE APPLICABLE TO ANY
     PRODUCTS WHICH HAVE SUFFERED ACCIDENT, MISUSE, NEGLECT, OR HAVE BEEN
     MODIFIED WITHOUT LONATI'S CONSENT.

          (f)  Neither Speizman, nor any agent, representative or designee of
     Speizman shall make any warranties, representations or guarantees to any
     person, either orally or in writing, with respect to the Products, or in
     the name of, or on behalf of Lonati without Lonati's prior written
     consent."

     7.   Miscellaneous. This Amendment shall be binding upon and shall inure to
          -------------
the benefit of the parties hereto and their respective successors and permitted
assigns. Except as modified by this Amendment, the Original Agreement shall
remain in full force and effect and is hereby ratified and confirmed.

     8.   Counterparts. This Amendment may be executed by the parties in any
          ------------
number of counterparts each of which shall be deemed to be an original document,
but all of which taken together shall constitute one and the same document or
agreement.



                                       8

<PAGE>

     IN WITNESS WHEREOF, the parties hereto, through their duly authorized
officers, have executed this First Amendment to Agency Agreement as of the date
first written above.

                                     LONATI, S.p.A.


                                     By: /s/ Ettore Lonati
                                         ---------------------------------------
                                             Ettore Lonati, Director

                                     SPEIZMAN INDUSTRIES, INC.


                                     By: /s/ Robert S. Speizman
                                         ---------------------------------------
                                             Robert S. Speizman, President

ATTEST:


/s/ John C. Angelella
--------------------------------
John C. Angelella, Secretary

                                        9

<PAGE>

                                    EXHIBIT A

Single Cylinder Sock Machines:
-----------------------------

-   L442J
-   L454
-   L474
-   L454 J open toe
-   L474 J open toe
-   L454 J closed toe
-   L474 J closed toe
-   FL54 J open toe
-   FL74 J open toe
-   FL54 J closed toe
-   FL74 J closed toe
-   L462
-   L472
-   L462 K
-   L472 K
-   L462 6 colors
-   L472 6 colors
-   L462 6 C closed toe
-   L472 6 C closed toe
-   FL 62 6C open toe
-   FL 72 6C open toe
-   FL 62C closed toe
-   FL 72C 6C closed toe


Double Cylinder Machines:
------------------------

-   Bravo DL
-   Bravo J
-   Bravo J 3
-   Bravo J 6
-   Bravo R
-   Bravo R 6
-   Bravo RL
-   Bravo RL 6
-   Bravo 1

                                       10