<PAGE>
 
                                                                    EXHIBIT 10.2

                               FIRST AMENDMENT TO
                    CONVERTIBLE DEBENTURE PURCHASE AGREEMENT
                                       AND
                              CONVERTIBLE DEBENTURE

                  THIS FIRST AMENDMENT TO CONVERTIBLE DEBENTURE PURCHASE
AGREEMENT AND CONVERTIBLE DEBENTURE (this "Amendment") is made as of May 30,
                                           ---------
1997, between The Right Start, Inc., a California corporation (the "Company")
                                                                    -------
and Strategic Associates, L.P., a Delaware limited partnership (the
"Purchaser").
 ---------

                                R E C I T A L S:

                  A. The Company and the Purchaser entered into that certain
Convertible Debenture Purchase Agreement dated as of October 11, 1996 (the
"Agreement") providing for the sale by the Company to the Purchaser of the
 ---------
Company's Convertible Debenture dated October 11, 1996 (the "Debenture"). All
                                                             ---------
capitalized terms used, and not otherwise defined, in this Amendment shall have
the meanings set forth in the Agreement;


                  B. The Company issued the Debenture to the Purchaser on
October 11, 1996;

                  C. The Company and the Purchaser desire to amend the Agreement
and the Debenture as set forth below for good and valuable consideration, the
receipt and sufficiency of which are hereby acknowledged.

                  Accordingly, the parties hereby agree to amend and modify the
Agreement and to restate the Debenture as set forth below:

                  1.       The last sentence of Section 2.01 of the Agreement
shall be amended to read in its entirety as follows:

                           "The Debenture, from and after the effectiveness of
         the First Amendment to this Agreement dated as of May 30, 1997, shall
         have a conversion price of $4.00 per share of Common Stock, subject to
         adjustment in certain circumstances."

                  2.       Section 5.01 of the Agreement shall be amended by
adding the following provision at the end of such section:

                           "(l) Change of Control. If at any time there is a
                                -----------------
         Change of Control of the Company then the Company shall, immediately
         following the occurrence of any such event, send a notice to Purchaser
         offering to repurchase the Debenture (or at each Purchaser's option,
         any portion thereof) at the par amount thereof, plus interest accrued
         and unpaid on the Debenture to the date of such repurchase. If the
         Purchaser desires to accept such offer in whole or in part, the
         Purchaser must advise the Company of such acceptance within 30 days of
         the date of receiving such notice. The Company shall then repurchase
         the Debenture or portion thereof so tendered for repurchase by the
         Purchaser by paying the purchase price to the Purchaser (or any person
         or persons designated by the Purchaser in such acceptance notice), in
         immediately available funds, within five days of the Company's receipt
         of the Purchaser's acceptance notice. For purposes of this
<PAGE>
 
         covenant (l), "Change of Control" shall mean an event or series of
         events by which (1) Kayne Anderson ceases to beneficially own (as
         beneficial ownership is defined in Rule 13d-3 under the Securities
         Exchange Act of 1934, as amended (the "Exchange Act")) and control,
         directly or indirectly, at least twenty-five percent (25%) of the
         issued and outstanding shares of each class of capital stock of the
         Company entitled (without regard to the occurrence of any contingency)
         to vote for the election of a majority of the members of the board of
         directors of the Company; (2) any person or group (as defined in Rule
         13d-1 of the Exchange Act), other than a group which includes Kayne
         Anderson, who obtains beneficial ownership (as defined in Rule 13d-3 of
         the Exchange Act) or control of a majority of the securities of the
         Company ordinarily having the right to vote in the election of
         directors; (3) during any two year period commencing on the date
         hereof, individuals who at the beginning of such period constituted the
         Board of Directors cease for any reason to constitute a majority of the
         Board of Directors; (4) any sale, lease, exchange or other transfer (in
         one transaction or a series of related transactions) of all, or
         substantially all, the assets of the Company; (5) the merger or
         consolidation of the Company with another corporation with the effect
         that immediately after such transaction any beneficial owner of the
         Company shall have become the beneficial owner of securities of the
         corporation surviving such merger or consolidation representing a
         majority of the combined voting power of the outstanding securities of
         the surviving corporation ordinarily having the right to vote in the
         election of directors; or (6) the adoption of a plan leading to the
         liquidation or dissolution of the Company. For purposes of this
         covenant (l), "Kayne Anderson" means Kayne Anderson Investment
         Management, Inc., KAIM Non-Traditional, L.P., Kayne Anderson
         Non-Traditional Investments, L.P., Kayne Anderson Offshore Limited,
         ARBCO Associates, L.P., Offense Group Associates, L.P., and Opportunity
         Associates, L.P. and each of their affiliates.

                  3. On the date hereof, the Debenture as issued shall be
cancelled and the Company shall deliver to the Purchaser a replacement debenture
identical in all respects to the Debenture, except that Section 2.1 of the
Debenture shall have been amended to read in its entirety as follows:

                           "2.1 Right of Conversion. Subject to and in
                                -------------------
         compliance with the provisions of this Section 2, the Holder shall have
         the right, at the Holder's option, at any time, and before the date on
         which the entire principal amount hereof, all accrued and unpaid
         interest hereon, and all other amounts payable to the Holder hereunder
         or under the Purchase Agreement have been paid in full (the "Expiration
         Date"), to convert the principal amount of this Convertible Debenture,
         or any portion thereof, into the number of fully paid and nonassessable
         shares of Common Stock, no par value, of the Company determined by
         dividing the principal amount so converted by the purchase price per
         share of $4.00, as adjusted from time to time as hereinafter provided
         (the "Conversion Price").

                  4. The Agreement shall remain in full force and effect as
amended hereby and all parties hereto hereby ratify and affirm the Agreement as
so amended.

                                      -2-
<PAGE>
 
                  5. Two or more duplicate originals of this Amendment may be
signed by the parties, each of which shall be an original but all of which
together shall constitute one and the same original.

                  6. This Amendment, the rights and obligations of the parties
hereto, and any claims or disputes relating thereto, shall be governed by and
construed in accordance with the laws of the State of Delaware (excluding the
choice of law rules thereof).

                  IN WITNESS WHEREOF, this Amendment is executed as of the day
and year first above written.


                                 THE RIGHT START, INC.



                                 By:
                                       -------------------------
                                       Jerry R. Welch
                                       President and Chief Executive Officer



                                 STRATEGIC ASSOCIATES, L.P.

                                 By:   Cahill, Warnock & Company, LLC,
                                       its general partner



                                 By:
                                       -------------------------
                                       David Warnock

                                      -3-