<PAGE>   1
                                                                    EXHIBIT 10.1

                       FOURTH AMENDMENT TO NOTE AGREEMENT

Reference is made to the Note Agreement dated as of March 15, 1991, as amended
(the "Note Agreement") between Dreyer's Grand Ice Cream, Inc. (the "Company"),
and Massachusetts Mutual Life Insurance Company, Connecticut Mutual Life
Insurance Company (now Massachusetts Mutual Life Insurance Company), the
Equitable Life Assurance Society of the United States, Transamerica Occidental
Life Insurance Company, and MML Pension Insurance Company (now UniCare Life and
Health Insurance Company, whose interest in the Notes has been sold to
Massachusetts Mutual Life Insurance Company) (together, the "Holders").

WHEREAS, the Company has requested relief from the provisions of Sections 5.8
and 5.8A of the Note Agreement;

WHEREAS, the Company represents and warrants to the Holders that, after giving
effect to this Amendment, no Default or Event of Default shall be outstanding
under the Note Agreement; and

WHEREAS, the Company and the Holders are desirous of amending the Note Agreement
on the terms and conditions set forth below.

NOW THEREFORE, the Company and the Holders agree that the Note Agreement is
hereby amended as follows:

1. Sections 5.8 and 5.8A of the Note Agreement are hereby deleted and replaced
in their entirety with the following:

            "Section 5.8. Minimum Consolidated Net Worth. The Company will not,
at any time, permit Consolidated Net Worth to be less than the sum of (a)
$170,000,000 plus (b) an aggregate amount equal to 25% of Consolidated Net
Income (but, in each case, only if a positive number) for each completed fiscal
year beginning with the fiscal year ending December 31, 1996."

2. The definition of "Restricted Investments" contained in Section 8 of the Note
Agreement is hereby deleted and replaced with the following:

           ""Restricted Investments" means all Investments except (i)
Investments in property to be used in the ordinary course of business; (ii)
Investments in Restricted Subsidiaries; (iii) Investments in obligations,
maturing within three years, issued by or guaranteed by the United States of
America or an agency thereof; (iv) Investments in municipal securities, maturing
within three years, which are rated in one of the top two rating classifications
by at least one National Rating Agency; (v) Investments in certificates of
deposit or banker's acceptances issued by Bank of America N.T. & S.A. or other
commercial banks, which are rated in one of the top two rating classifications
by at least one National Rating Agency; (vi) Investments consisting of
repurchase arrangements with banks meeting the requirements of clause (v) of
this definition; (vii) Investments in commercial paper, maturing




<PAGE>   2

within 270 days, rated in one of the top two rating classifications by at least
one National Rating Agency; (viii) Investments in money market instrument
programs which have a policy of maintaining net asset value of at least $1.00
and are classified as current assets in accordance with generally accepted
accounting principles in effect at the time; (ix) loans or advances in the
ordinary course of business to officers, directors and employees for expenses
incidental to carrying on the business of the Company or any Restricted
Subsidiary; and (x) any other Investments listed on Exhibit X attached hereto."

3. The following new definitions are hereby added to Section 8 of the Note
Agreement, in the appropriate alphabetical order:

           "Consolidated Net Worth" means, at any time, without duplication, the
consolidated stockholders' equity of the Company and its Restricted
Subsidiaries, as determined in accordance with generally accepted accounting
principles in effect at the time, plus the Stated Value of the Company's Series
B Preferred Stock in existence on June 6, 1996, and any Series A Preferred Stock
into which such existing Series B Preferred Stock may be converted, less the
amount of Restricted Investments of the Company and its Restricted Subsidiaries
in excess of 15% of consolidated stockholders' equity of the Company and its
Restricted Subsidiaries.

           "National Rating Agency" means each of Moody's Investor Service,
Inc., Standard and Poor's Ratings Group, a division of McGraw Hill, Inc., and
Duff & Phelps Credit Rating Co."

           "Preferred Stock" means any class of capital stock of a corporation
that is preferred over any other class of capital stock of such corporation as
to the payment of dividends or the payment of any amount upon liquidation or
dissolution of such corporation."

           "Series A Preferred Stock" means the Company's Series A Convertible
Preferred Stock.

           "Series B Preferred Stock" means the Company's Series B Convertible
Preferred Stock.

           "Stated Value" means, at any time with respect to any Preferred
Stock, the stated or liquidation value of such Preferred Stock without giving
effect to accrued dividends."

4. The capitalized terms used herein shall have the respective meanings
specified in the Note Agreement unless otherwise defined herein or if the
context shall otherwise require.

5. Except as amended herein, the terms and provisions of the Note Agreement are
hereby ratified, confirmed and approved in all respects.








<PAGE>   3


6. The effectiveness of this Amendment is expressly conditioned on the accuracy
of the Company's representations and warranties set forth above.

7. This document shall be dated as of June 10, 1997.

ACCEPTED AND AGREED TO:

DREYER'S GRAND ICE CREAM, INC.              MASSACHUSETTS MUTUAL LIFE
                                            INSURANCE COMPANY


/s/ William C. Collett                      /s/ John B. Joyce
-----------------------------               -----------------------------
By:  William C. Collett                     By:  John B. Joyce
Its:  Treasurer                             Its:  Managing Director

THE EQUITABLE LIFE ASSURANCE                TRANSAMERICA OCCIDENTAL
SOCIETY OF THE UNITED STATES                LIFE INSURANCE COMPANY


                                            /s/ John M. Casparian
-----------------------------               -----------------------------
By:                                         By:  John M. Casparian
Its:                                        Its:  Investment Officer