<PAGE>   1
                                                                   Exhibit 10.34

                               AMENDMENT NO. ONE
                                       TO
                          LOAN AND SECURITY AGREEMENT

This Amendment No. One to the Loan and Security Agreement dated JANUARY 29,     
1999 (this "Amendment") is made as of the 29TH day of JANUARY, 1999, by and
between General Electric Capital Business Asset Funding Corporation ("Lender")
and Michael Anthony Jewelers, Inc. ("Borrower") with respect to the following
facts:

         a. Lender and Borrower are parties to that certain Loan and Security
Agreement of even date herewith (the "Agreement").

         b. Lender and Borrower wish to amend the Agreement as provided in this
Amendment.

NOW, THEREFORE, for good and valuable consideration, the receipt and sufficiency
of which are hereby acknowledged by Lender and Borrower, Lender and Borrower
hereby agree as follows:

         1. Section 5(e) is hereby amended by adding the following sentence
after the first sentence of such section:

            The foregoing sentence shall not be construed to prohibit Borrower
            from seeking and, if granted, complying with extensions of tax
            filing or payment deadlines, provided the same is permitted under
            applicable law and does not result in any lien on any Equipment.

         2. Section 5(f) of the Agreement is hereby amended by adding the
following language at the end of such section:

            except for any lender's lien on any Equipment if (but only if) such
            lien is (i) disclosed to Lender in writing by Borrower prior to the
            closing of the Loan for such Equipment and (ii) released
            simultaneously with such closing (any such lien, if so disclosed and
            released, being referred to herein as a "Temporary Lien").

         3. Section 6(c) of the Agreement is hereby amended by adding the
following language before the word "Borrower" where such word first appears in
such section:

            and except for any Temporary Liens,

         4. Section 6(c) of the Agreement is hereby further amended by adding 
the following language at the end of such section:

            ; provided, that Borrower may sell or otherwise dispose of any of
            the Equipment if, but only if, Borrower is not then in default and,
            prior to such sale or other disposition, Borrower either (i) repays
            to Lender the portion of the remaining balance of the Loan
            attributable to such Equipment (such amount to be approved by
            Lender), plus the prepayment premium thereon (calculated in
            accordance with the formula set forth in the Term Note for such
            Loan) or (ii) grants to Lender a security interest in other
            equipment of similar type and value that is (A) owned by Borrower
            free and clear of all liens or other claims or interests (other
            than the subordinate liens, other claims or interests (if any) that
            were permitted on the disposed Equipment) and (B) otherwise in
            compliance with and fully subject to the requirements of the
            applicable Loan Documents, and Borrower executes and delivers to
            Lender such financing statements and other documents as Lender
            shall request to perfect and protect its first priority security
            interest in such replacement equipment; further provided, that,
            within any one-year period, (x) Borrower may only so dispose of
            Equipment a maximum of two times and (y) the total remaining
            balance of the Loan attributable to the Equipment disposed in such
            year shall not exceed $200,000.

         5. Section 7(d) of the Agreement is hereby amended by adding the
following language at the end of such section:

            provided, that the releases for any Temporary Liens may be delivered
            simultaneously with the closing of the Loan for the Equipment that
            is the subject of such Temporary Liens.

<PAGE>   2

         6. Section 7(e) of the Agreement is hereby amended by adding the
following language at the end of such section:

         ; provided that, in the case of any Equipment subject to any
         Temporary Lien, the financing statement(s) to be filed thereon by or
         for Lender shall have been duly executed and delivered by Borrower to
         Lender but shall not be filed prior to closing the Loan related to such
         Equipment.

         7. Section 9(d) of the Agreement is hereby amended in its entirety to
read as follows:

         Sell, lease or otherwise dispose of the Collateral at public or private
         sale, with or without having the Collateral at the place of sale, and
         upon terms and in such manner as Lender may determine (and Lender may
         be a purchaser at any sale), Borrower hereby agreeing that, for a
         period not to exceed 120 days, and without any charge to Lender, (i)
         Lender may store the Collateral at any premises at which the same is
         located and attempt to sell the Collateral therefrom and (ii) Borrower
         shall ensure that electricity (including back-up electricity for
         security purposes) and access to any safes are provided; and

IN WITNESS WHEREOF, Lender and Borrower have caused their respective duly
authorized representatives to execute and deliver this Amendment as of the day
and year first above written.

GENERAL ELECTRIC CAPITAL BUSINESS               MICHAEL ANTHONY JEWELERS, INC.
ASSET FUNDING CORPORATION

By                                              By /s/ MICHAEL A. PAOLERCIO
   ------------------------------                  -----------------------------
Its                                             Its      TREASURER
   ------------------------------                  -----------------------------

                                        2
<PAGE>   3

                                         SUPPLEMENTAL SECURITY AGREEMENT NO. ONE
                                                           LOAN #001-0003281-001

This Supplemental Security Agreement is executed by Michael Anthony Jewelers,
Inc. ("Borrower") pursuant to the terms of a Loan and Security Agreement dated
January 29, 1999 between Borrower and General Electric Capital  Business Asset
Funding Corporation ("Lender"). All capitalized terms used herein that are not
otherwise defined herein shall have the respective meanings given to such terms
in the Loan and Security Agreement.

     In order to provide security for the payment and performance of Borrower's
obligations under the Loan Documents, Borrower has granted to Lender a first
priority security interest in the Collateral. In addition to said grant,
Borrower intends by this Supplemental Security Agreement to grant to Lender a
first priority security interest in the items of Equipment identified herein.

     1. To further secure the payment and performance of all of Borrower's
obligations to Lender under the Loan Documents, Borrower hereby grants to Lender
a first priority security interest in the items of Collateral described below,
including all present and future additions, attachments and accessories thereto,
all substitutions therefor and replacements thereof and all proceeds thereof,
including all proceeds of insurance:

<TABLE>
<CAPTION>
Qty.     Model/Mfr.                 Description                   Serial No.                Cost or Appraised Value
----     ----------                 -----------                   ----------                -----------------------
<S>      <C>        <C>                                           <C>                       <C> 
                    Jewelery molds, casts, dyes, tools, machinery                                  $10,444,444.46
                    and equipment.
</TABLE>

2. Borrower hereby (a) affirms that the representations and warranties set forth
in Section 5 of the Loan and Security Agreement are true and correct as of the
date hereof; (b) represents and warrants that Lender has a first priority
security interest in the Collateral; and (c) represents and warrants that the
above described equipment will be maintained at the following locations:

<TABLE>
<S>                              <C>                        <C>                       <C>                     
  115 SOUTH MACQUESTEN PARKWAY,  50 S. MACQUESTEN PARKWAY,  60S. MACQUESTEN PARKWAY,  70 S. MACQUESTEN PARKWAY
  MT. VERNON NY 10550            MT. VERNON, NY 10550       MT. VERNON, NY 10550      MT. VERNON, NY 10550
</TABLE>

3. The Loan Amount for loans to be made pursuant to this Supplemental Security
Agreement is $10,444,444.46

4. The Commitment Expiration Date for loans to be made pursuant to this
Supplemental Security Agreement is JANUARY 30,1999.

5. The amount of liability insurance required to be maintained by Borrower
pursuant to Section 6(d) of the Loan and Security Agreement is
$_______________________________.

6. All of the terms and provisions of the Loan and Security Agreement are hereby
incorporated in and made a part of this Supplemental Security Agreement to the
same extent as if fully set forth herein.


     In witness whereof, Borrower has executed and delivered this Supplemental
Security Agreement this 29 day of January 1999.
                        --        ------------

                                Borrower:         Michael Anthony Jewelers, Inc.
                                                  ------------------------------
                                By:               /s/ Michael A. Paolercio
                                                  ------------------------------
                                (Print Name):     Michael A. Paolercio
                                                  ------------------------------
                                Title:            Treasurer
                                                  ------------------------------