<PAGE>
 
                                                                    EXHIBIT 10.1

                               FIRST AMENDMENT TO
                    CONVERTIBLE DEBENTURE PURCHASE AGREEMENT
                                       AND
                              CONVERTIBLE DEBENTURE

                  THIS FIRST AMENDMENT TO CONVERTIBLE DEBENTURE PURCHASE
AGREEMENT AND CONVERTIBLE DEBENTURE (this "Amendment") is made as of May 30,
                                           ---------
1997, between The Right Start, Inc., a California corporation (the "Company")
                                                                    -------
and Cahill Warnock Strategic Partners Fund, L.P., a Delaware limited partnership
(the "Purchaser").
      ---------

                                R E C I T A L S:

                  A. The Company and the Purchaser entered into that certain
Convertible Debenture Purchase Agreement dated as of October 11, 1996 (the
"Agreement") providing for the sale by the Company to the Purchaser of the
 ---------
Company's Convertible Debenture dated October 11, 1996 (the "Debenture"). All
                                                             ---------
capitalized terms used, and not otherwise defined, in this Amendment shall have
the meanings set forth in the Agreement and the Debenture;


                  B. The Company issued the Debenture to the Purchaser on
October 11, 1996; and

                  C. The Company and the Purchaser desire to amend the Agreement
and the Debenture as set forth below for good and valuable consideration, the
receipt and sufficiency of which are hereby acknowledged.

                  Accordingly, the parties hereby agree to amend and modify the
Agreement and to restate the Debenture as set forth below:

                  1.       The last sentence of Section 2.01 of the Agreement
shall be amended to read in its entirety as follows:

                           "The Debenture, from and after the effectiveness of
         the First Amendment to this Agreement dated as of May 30, 1997, shall
         have a conversion price of $4.00 per share of Common Stock, subject to
         adjustment in certain circumstances."

                  2.       Section 5.01 of the Agreement shall be amended by
adding the following provision at the end of such section:

                           "(l) Change of Control. If at any time there is a
                                -----------------
         Change of Control of the Company then the Company shall, immediately
         following the occurrence of any such event, send a notice to Purchaser
         offering to repurchase the Debenture (or at each Purchaser's option,
         any portion thereof) at the par amount thereof, plus interest accrued
         and unpaid on the Debenture to the date of such repurchase. If the
         Purchaser desires to accept such offer in whole or in part, the
         Purchaser must advise the Company of such acceptance within 30 days of
         the date of receiving such notice. The Company shall then repurchase
         the Debenture or portion thereof so tendered for repurchase by the
         Purchaser by paying the purchase price to the Purchaser (or any person
         or persons designated by the
<PAGE>
 
         Purchaser in such acceptance notice), in immediately available funds,
         within five days of the Company's receipt of the Purchaser's acceptance
         notice. For purposes of this covenant (l), "Change of Control" shall
         mean an event or series of events by which (1) Kayne Anderson ceases to
         beneficially own (as beneficial ownership is defined in Rule 13d-3
         under the Securities Exchange Act of 1934, as amended (the "Exchange
         Act")) and control, directly or indirectly, at least twenty-five
         percent (25%) of the issued and outstanding shares of each class of
         capital stock of the Company entitled (without regard to the occurrence
         of any contingency) to vote for the election of a majority of the
         members of the board of directors of the Company; (2) any person or
         group (as defined in Rule 13d-1 of the Exchange Act), other than a
         group which includes Kayne Anderson, who obtains beneficial ownership
         (as defined in Rule 13d-3 of the Exchange Act) or control of a majority
         of the securities of the Company ordinarily having the right to vote in
         the election of directors; (3) during any two year period commencing on
         the date hereof, individuals who at the beginning of such period
         constituted the Board of Directors cease for any reason to constitute a
         majority of the Board of Directors; (4) any sale, lease, exchange or
         other transfer (in one transaction or a series of related transactions)
         of all, or substantially all, the assets of the Company; (5) the merger
         or consolidation of the Company with another corporation with the
         effect that immediately after such transaction any beneficial owner of
         the Company shall have become the beneficial owner of securities of the
         corporation surviving such merger or consolidation representing a
         majority of the combined voting power of the outstanding securities of
         the surviving corporation ordinarily having the right to vote in the
         election of directors; or (6) the adoption of a plan leading to the
         liquidation or dissolution of the Company. For purposes of this
         covenant (l), "Kayne Anderson" means Kayne Anderson Investment
         Management, Inc., KAIM Non-Traditional, L.P., Kayne Anderson
         Non-Traditional Investments, L.P., Kayne Anderson Offshore Limited,
         ARBCO Associates, L.P., Offense Group Associates, L.P., and Opportunity
         Associates, L.P. and each of their affiliates.

                  3. On the date hereof, the Debenture as issued shall be
cancelled and the Company shall deliver to the Purchaser a replacement debenture
identical in all respects to the Debenture, except that Section 2.1 of the
Debenture shall have been amended to read in its entirety as follows:

                           "2.1 Right of Conversion. Subject to and in
                                -------------------
         compliance with the provisions of this Section 2, the Holder shall have
         the right, at the Holder's option, at any time, and before the date on
         which the entire principal amount hereof, all accrued and unpaid
         interest hereon, and all other amounts payable to the Holder hereunder
         or under the Purchase Agreement have been paid in full (the "Expiration
         Date"), to convert the principal amount of this Convertible Debenture,
         or any portion thereof, into the number of fully paid and nonassessable
         shares of Common Stock, no par value, of the Company determined by
         dividing the principal amount so converted by the purchase price per
         share of $4.00, as adjusted from time to time as hereinafter provided
         (the "Conversion Price").

                  4. The Agreement shall remain in full force and effect as
amended hereby and all parties hereto hereby ratify and affirm the Agreement as
so amended.

                                      -2-
<PAGE>
 
                  5. Two or more duplicate originals of this Amendment may be
signed by the parties, each of which shall be an original but all of which
together shall constitute one and the same original.

                  6. This Amendment, the rights and obligations of the parties
hereto, and any claims or disputes relating thereto, shall be governed by and
construed in accordance with the laws of the State of Delaware (excluding the
choice of law rules thereof).

                  IN WITNESS WHEREOF, this Amendment is executed as of the day
and year first above written.

                                 THE RIGHT START, INC.



                                 By:
                                       ------------------------
                                       Jerry R. Welch
                                       President and Chief Executive Officer



                                 CAHILL, WARNOCK STRATEGIC PARTNERS FUND, L.P.

                                 By:   Cahill, Warnock Strategic Partners,
                                       L.P., its general partner



                                 By:
                                       ------------------------
                                       David Warnock

                                      -3-