<PAGE>   1
                                                                  EXHIBIT 4.1(b)



================================================================================



                            JEF HOLDING COMPANY, INC.


                              JEFFERIES GROUP, INC.


                                       AND


                              THE BANK OF NEW YORK,

                                     TRUSTEE

                           ---------------------------



                          SECOND SUPPLEMENTAL INDENTURE


                           DATED AS OF MARCH 17, 1999



                           ---------------------------



                                  $100,000,000

                          7 1/2% SENIOR NOTES DUE 2007

                          ----------------------------

              SUPPLEMENTING THE INDENTURE OF JEFFERIES GROUP, INC.
            DATED AS OF AUGUST 18, 1997, AS AMENDED ON MARCH 15, 1999



================================================================================

<PAGE>   2

        SECOND SUPPLEMENTAL INDENTURE, dated as of March 17, 1999, between JEF
Holding Company, Inc., a Delaware corporation (the "New JEF"), Jefferies Group,
Inc., a Delaware Corporation ("Group") and The Bank of New York, a New York
banking corporation, as trustee (the "Trustee").

        WHEREAS, Group and the Trustee executed and delivered that certain
indenture, dated as of August 18, 1997 and as amended on March 15, 1999
(collectively, the "Indenture"), concerning Group's 7 1/2% Senior Notes due 2007
(the "Securities"). Any capitalized terms used herein and not otherwise defined
shall have the meanings given thereto in the Indenture.

        WHEREAS, as of the date hereof, New JEF is a wholly owned subsidiary of
Group.

        WHEREAS, Group has taken appropriate corporate action to effect the
Spin-Off prior to the ITGI Merger, as of April 27, 1999.

        WHEREAS, New JEF, Group and the Trustee, by appropriate corporate
action, have determined to supplement the Indenture in the manner described
below and all acts and proceedings required by law, by the Indenture, and by the
Certificate of Incorporation and the Bylaws of New JEF and Group necessary to
authorize and constitute this Second Supplemental Indenture a valid and binding
agreement in accordance with the terms hereof, have been done and taken.

        NOW, THEREFORE, in consideration of the foregoing, each of New JEF and
Group covenants and agrees with the Trustee, for the equal and proportionate
benefit of the respective holders from time to time of the Securities, as
follows:

        1. Assumption. New JEF hereby agrees that, upon completion of the
Transfers in accordance with the Distribution Agreement (excluding any transfers
subject to Section 3.01(e) of the Distribution Agreement), New JEF shall succeed
to, be substituted for, and exercise every right and power of Group under the
Indenture, as amended hereby, and shall assume, without executing any further
document or instrument, all of the obligations of Group pursuant to the
Indenture, as amended hereby, and pursuant to the Securities. Upon the
completion of the Transfers in accordance with the Distribution Agreement
(excluding any transfers subject to Section 3.01(e) of the Distribution
Agreement), and the succession and substitution set forth in the preceding
sentence, Group shall thereby be relieved of the performance and observance of
all obligations and covenants of the Indenture, as amended hereby, and of the
Securities, including but not limited to the obligation to make payment of the
principal of and interest, if any, on all the Securities.

        2. Full Force and Effect; Operative Effect of Amendment through Second
Supplemental Indenture. The Indenture, as amended by this Second Supplemental
Indenture, shall be in full force and effect as of the date hereof; provided,
however, the provisions of Paragraph 1 shall only become operative in connection
with the completion of the Transfers in



                                      - 1 -

<PAGE>   3

accordance with the Distribution Agreement (excluding any transfers subject to
Section 3.01(e) of the Distribution Agreement), if such completion of Transfers
(excluding the exceptions noted in the preceding parenthetical) is so effected
prior to May 1, 1999.

        3. Governing Law. This Second Supplemental Indenture shall be governed
by and construed in accordance with the laws of the State of New York.

        4. Duplicate Originals. This Second Supplemental Indenture may be
executed in any number of counterparts and by the parties hereto in separate
counterparts, each of which when so executed shall be deemed to be an original
and all of which taken together shall constitute one and the same agreement.

        IN WITNESS WHEREOF, the parties hereto have caused this Second
Supplemental Indenture to be duly executed as of the date first written above.

                                        JEF HOLDING COMPANY, INC.



                                        By:  /s/ Jerry M. Gluck
                                            ------------------------------------
                                            Name:  Jerry M. Gluck
                                            Title: Secretary



                                        JEFFERIES GROUP, INC.



                                        By:  /s/ Clarence T. Schmitz
                                            ------------------------------------
                                            Name:  Clarence T. Schmitz
                                            Title: Executive Vice President



                                        THE BANK OF NEW YORK,
                                        as Trustee



                                        By:  /s/ Mary La Gumina
                                            ------------------------------------
                                            Name:  Mary La Gumina
                                            Title: Assistant Vice President



                                      - 2 -