<PAGE>   1
                                                                    Exhibit 10.1


April 24, 2001

Wallace G.K. Chin
President
Kamehameha Activities Association
567 South King Street, Suite 301
Honolulu, HI 96813

The Trustees of the Estate of Bernice Pauahi Bishop
P.O. Box 3466
567 South King Street, Suite 200
Honolulu, HI 96813

Dear Mr. Chin and Bishop Estate Trustees:

         This letter (this "Letter Agreement") sets forth the agreement among
Kamehameha Activities Association ("KAA" or "Knight"), The Trustees of the
Estate of Bernice Pauahi Bishop (the "Bishop Estate") and The Goldman Sachs
Group, Inc. ("GS Inc.") with respect to certain matters. Reference is made to
(i) the Subscription Agreement, dated as of April 24, 1992 (as amended from time
to time, the "1992 Subscription Agreement"), among the Bishop Estate, Pauahi
Holdings Corporation ("Knight's Parent") and Royal Hawaiian Shopping Center,
Inc. ("RHSC") (which transferred all its interests in The Goldman Sachs Group,
L.P. ("Group LP") to KAA as set forth in an Assumption Agreement, dated July 15,
1998 (the "Assumption Agreement"), between KAA and RHSC for the benefit of Group
LP), (ii) the Subscription Agreement, dated as of November 21, 1994 (as amended
from time to time, the "1994 Subscription Agreement" and, collectively with the
1992 Subscription Agreement, the "Subscription Agreements"), among the Bishop
Estate, Knight's Parent and RHSC, on the one hand, and Group LP, on the other
hand, (iii) the Registration Rights Agreement, dated as of April 24, 1992 (as
amended from time to time, the "Registration Rights Agreement"), between RHSC
and Group LP, as amended by Amendment No. 1 thereto dated November 21, 1994,
(iv) the Letter Agreement, dated March 15, 1999 (as amended from time to time,
the "1999 Letter Agreement"), between KAA and Group LP, (v) the Amendment to the
Letter Agreement, dated April 30, 1999 (the "1999 Letter Agreement Amendment"),
among KAA, the Bishop Estate, Group LP and GS Inc., (vi) the Amendment to the
Letter Agreement, dated August 1, 2000 (the "2000 Letter Agreement Amendment"),
among KAA, the Bishop Estate and GS Inc. and (vii) the Amendment to the Letter
Agreement, dated February 6, 2001 (the "2001 Letter Agreement Amendment" and,
together with the 1999 Letter Agreement, the 1999 Letter Agreement Amendment and
the 2000 Letter Agreement Amendment, the "Letter Agreement"), among KAA, the
Bishop Estate and GS Inc. KAA has assumed all of RHSC's rights and obligations
under the Subscription Agreements and the

<PAGE>   2
Kamehameha Activities Association
April 24, 2001
Page 2


Registration Rights Agreement as set forth in the Assumption Agreement. This
Letter Agreement shall be a modification of and an amendment to the Subscription
Agreements, the Registration Rights Agreement and the Letter Agreement, to the
extent set forth herein.

         GS Inc. has filed with the Securities and Exchange Commission (the
"Commission") a registration statement on Form S-3 (the "Registration
Statement") with respect to 4,032,513 shares (the "Shares") of the common stock,
par value $.01 per share (the "Common Stock"), of GS Inc. beneficially owned by
KAA, and GS Inc. has been informed by the Commission that the Commission is
prepared to declare the Registration Statement effective. Accordingly, the
parties hereby agree as follows:


         1.       Common Stock to Be Sold by KAA.


                  (a) GS Inc. agrees to request that the Commission declare the
         Registration Statement effective as of 4 p.m., Eastern daylight time,
         on April 25, 2001.


                  (b) KAA agrees that when the Registration Statement is
         declared effective by the Commission on or before May 10, 2001, KAA
         will be deemed to have delivered a Disposition Notice (as defined in
         the Registration Rights Agreement) during the 12-month period following
         the second anniversary of the IPO Date (as defined in the Letter
         Agreement). So long as the Registration Statement is declared effective
         by the Commission on or before May 10, 2001, GS Inc. hereby waives its
         rights under the Registration Rights Agreement to deliver any Delay
         Notice, Purchase Notice or MD Purchase Notice (each as defined in the
         Registration Rights Agreement) with respect to the deemed Disposition
         Notice.

                  (c) As soon as practicable after the Registration Statement is
         declared effective by the Commission, Goldman, Sachs & Co. will make an
         offer to purchase all of the Shares in a firm commitment underwriting
         managed solely by Goldman, Sachs & Co. (the "Offering"). KAA may elect
         to sell all or none (but not a portion) of the Shares in the Offering
         at the purchase price per Share specified by Goldman, Sachs & Co. KAA
         acknowledges and agrees that Goldman, Sachs & Co. will engage in
         marketing efforts similar to those for "block" transactions, but will
         not engage in a "road show" or other special marketing efforts.

<PAGE>   3
Kamehameha Activities Association
April 24, 2001
Page 3


                  (d) If KAA does not elect to sell the Shares in the Offering
         pursuant to Section 1(c), KAA may not deliver an additional Disposition
         Notice (the "Second Disposition Notice") until the second trading day
         immediately following the day on which GS Inc. publicly announces its
         earnings for the fiscal quarter ending May 25, 2001 (the "Earnings
         Release Date"). GS Inc. retains its rights under the Registration
         Rights Agreement to deliver a Delay Notice, Purchase Notice or MD
         Purchase Notice following the delivery of the Second Disposition
         Notice.

                  (e) KAA agrees that if it disposes of the Shares in accordance
         with Section 1(c), KAA shall be deemed to have disposed of shares of
         Common Stock constituting in the aggregate 20% of the Knight Original
         Block (as defined in the 1999 Letter Agreement) in the 12-month period
         following the second anniversary of the IPO Date.

                  (f) KAA agrees to execute a Power of Attorney substantially in
         the form of Annex A and the Bishop Estate agrees to execute a Power of
         Attorney substantially in the form of Annex B at or prior to the time
         of the execution by KAA or the Bishop Estate of any underwriting
         agreement relating to the Offering.

                  (g) If, despite the best efforts of GS Inc., the Registration
         Statement is not declared effective by the Commission on or before May
         10, 2001, KAA agrees that it will not deliver any Disposition Notice
         until the second trading day immediately following the Earnings Release
         Date.

                  (h) KAA agrees that it will pay all expenses of GS Inc. and
         KAA in connection with the Offering as provided by Section 3(d) of the
         Registration Rights Agreement.

         2.       Blackout Periods.

                  (a) Section 3 of the Registration Rights Agreement is hereby
         amended by adding the following subsection (e):

                           "(e) (i) On receipt of a Disposition Notice, the
                  Company may indicate, by written notice (a "Blackout Notice")
                  delivered to Knight within ten business days of receipt by the
                  Company of the Disposition Notice, that Knight may not dispose
                  of any shares of

<PAGE>   4
Kamehameha Activities Association
April 24, 2001
Page 4


                  Common Stock (A) during a quarterly blackout period with
                  respect to sales of Common Stock by employees of GS Inc. (an
                  "Employee Blackout Period") or (B) during a period of up to 90
                  days (a "Blackout Period") following the Company's receipt of
                  the Disposition Notice if in the judgment of the Company in
                  its sole discretion, distribution of the Common Stock would
                  materially interfere with any pending or proposed financing,
                  acquisition or other transaction involving the Company or
                  would be reasonably likely to require the disclosure of
                  material non-public information that the Company believes
                  would be detrimental to the Company, and Knight shall have no
                  right to proceed with the proposed distribution during the
                  Employee Blackout Period or Blackout Period (although Knight
                  may otherwise be afforded piggy-back rights pursuant to
                  Section 2). A Blackout Period shall terminate prior to its
                  expiration upon the judgment by the Company in its sole
                  discretion that the facts and circumstances giving rise to the
                  Blackout Period under clause (B) above are no longer present,
                  upon the completion or abandonment of the relevant financing,
                  acquisition or other transaction or upon the disclosure of the
                  relevant information to the public. The Company shall promptly
                  notify Knight of the expiration or earlier termination of any
                  Employee Blackout Period or Blackout Period.

                           (ii) Upon the expiration or termination of an
                  Employee Blackout Period or a Blackout Period, Knight shall be
                  entitled to deliver another Disposition Notice at such
                  expiration or termination if Knight still desires to effect a
                  disposition, and the Company will have no further right to
                  deliver a Delay Notice with respect to such intended
                  distribution unless Knight has not made such distribution
                  (other than due to the fault of the Company) within 90 days of
                  delivery to the Company of the second Disposition Notice."

                  (b) Section 3(b) of the Registration Rights Agreement is
         hereby amended by adding the words "or Blackout Notice" after the words
         "Delay Notice" in the second line thereof.

                  3. Future Sales. Section 4(a) of the 1999 Letter Agreement is
         hereby amended by replacing all references therein to "the third
         anniversary of the IPO Date" with "April 1, 2002". In addition, all
<PAGE>   5
Kamehameha Activities Association
April 24, 2001
Page 5


         references in the Letter Agreement to the 12-month period following the
         second anniversary of the IPO Date shall be deemed to refer to the
         period beginning on the second anniversary of the IPO Date and ending
         on March 31, 2002.

                  4. Agreements Otherwise Unimpaired. Except as expressly
         provided in this Letter Agreement, the Subscription Agreements, the
         Registration Rights Agreement, the Letter Agreement and any other
         agreements between or among the parties to this Letter Agreement shall
         not be modified, impaired or affected by the execution and delivery of
         this Letter Agreement.

                  5. Successors and Assigns. This Letter Agreement will be
         binding upon and inure to the benefit of and be enforceable by the
         respective successors and assigns of the parties hereto.

                  6. Governing Law. This Letter Agreement is being entered into
         and is intended to be performed in the State of New York and will be
         construed and enforced in accordance with and governed by the laws of
         the State of New York.

                  7. Counterparts. This Letter Agreement may be executed
         simultaneously in several counterparts, each of which is an original,
         but all of which together shall constitute one instrument.

<PAGE>   6
         Please indicate your agreement to the terms of this letter by signing
in the space provided below.


                                    THE GOLDMAN SACHS GROUP, INC.




                                    By:  /s/ Esta E. Stecher
                                       -----------------------------
                                         Name:  Esta E. Stecher
                                         Title: Executive Vice President and
                                                General Counsel


Accepted and Agreed to as
of the date first above
written:

KAMEHAMEHA ACTIVITIES ASSOCIATION



By: /s/ Wallace G.K. Chin
   ----------------------------------
   Name:  Wallace G.K. Chin
   Title: President





THE TRUSTEES OF THE ESTATE OF BERNICE PAUAHI BISHOP



By: /s/ James Douglas Keauhou Ing
   ----------------------------------

By: /s/ Robert Kalani Uichi Kihune
   ----------------------------------

By: /s/ Charles Nainoa Thompson
   ----------------------------------