<PAGE>

                                                                 EXHIBIT 10.16

                 FOURTH AMENDMENT TO FOURTH AMENDED AND RESTATED
                          REGISTRATION RIGHTS AGREEMENT

THIS FOURTH AMENDMENT, dated as of June 28, 2000 (the "Agreement"), to the
Fourth Amended and Restated Registration Rights Agreement dated as of April 11,
2000 as amended by the First Amendment thereto dated May 16, 2000, by and among
SPEECHWORKS INTERNATIONAL, INC., a Delaware corporation (the "Company") and the
stockholders of the Company named therein, as further amended by the Second
Amendment thereto dated June 5, 2000 (the "Registration Rights Agreement") by
and among the Company and the stockholders of the Company named therein, as
further amended by the Third Amendment thereto dated June 23, 2000 is by and
among the Company and Net2Phone, Inc. (the ""New Invest or") and the persons
named on the signature page hereto as Investors (collectively, the "Investors").
All other capitalized terms used herein and not otherwise defined have their
respective meanings set forth in the Registration Rights Agreement.

         WHEREAS, the New Investor is acquiring shares of the Common Stock,
$0.001 par value per share (the "Shares"), of the Company pursuant to the terms
of a certain Common Stock Purchase Agreement dated the date hereof (the
"Purchase Agreement");

         WHEREAS, the Purchase Agreement contemplates in Section 5 thereof that
the Registration Rights Agreement shall be amended to add the New Investor as a
party to the Registration Rights Agreement; and

         WHEREAS, the Investors who are signatories hereto hold in the aggregate
a sufficient number of shares of Registrable Stock to amend the Registration
Rights Agreement in accordance with Section 14 thereof to add the New Investor
as party thereto.

         NOW, THEREFORE, in consideration of the mutual covenants and agreements
set forth herein, the parties agree as follows:

         1. AMENDMENT TO SCHEDULE RRA (REVISED). From and after the date hereof,
SCHEDULE RRA to the Registration Rights Agreement is deleted and replaced by
SCHEDULE RRA (4TH REVISED), dated the date hereof and attached hereto and the
term "Investors" as used therein and herein shall mean and refer to all the
persons and entities identified on said SCHEDULE RRA (4TH REVISED).

         2. AMENDMENT TO DEFINITION OF "REGISTRABLE STOCK". The definition of
"Registrable Stock" set forth in Section 1 of the Registration Rights Agreement
is hereby deleted and replaced by the following definition:

"Registrable Stock" means (a) the Common Stock issued or issuable upon
conversion of the Series A Preferred Stock, Series B Preferred Stock, Series C
Preferred Stock, Series D Preferred Stock or Series E Preferred Stock, and owned
of record by any Investor or an


<PAGE>


Affiliate of any Investor; (b) all Common Stock now or hereafter owned of
record by any Investor which is acquired otherwise than upon conversion of
the Series A Preferred Stock, Series B Preferred Stock, Series C Preferred
Stock, Series D Preferred Stock or Series E Preferred Stock, so long as it is
held by any Investor or an Affiliate of the Investor, (c) all the shares of
Common Stock issued or issuable upon exercise of any warrant now or hereafter
held by Lighthouse, (do all the shares of Common Stock issued or issuable
upon exercise of any warrant now or hereafter held by InterVoice-Brite, Inc.,
and (e) any other shares of Common Stock issued in respect of such shares by
way of a stock dividend, or stock split or in connection with a combination
of shares, recapitalization, merger or consolidation or reorganization,
PROVIDED, HOWEVER, that shares of Common Stock shall only be treated as
Registrable Stock if and so long as they have not been (x) sold to or through
a broker or dealer or underwriter in a public distribution or a public
securities transaction, or (y) sold in a transaction exempt from the
registration and prospectus delivery requirements of the Securities Act under
Section 4(1) thereof so that all transfer restrictions and restrictive
legends with respect to such Common Stock are removed upon the consummation
of such sale.

         3. CONTINUED EFFECT. As amended hereby, the Registration Rights
Agreement is hereby ratified and confirmed and agreed to by all of the parties
hereto and continues in full force and effect.

         4. COUNTERPARTS. This Agreement may be executed in any number of
counterparts, each of which shall be deemed an original, but all of which
together shall constitute one and the same instrument.

         5. GOVERNING LAW. This Agreement shall be governed by, and construed in
accordance with the laws of the State of Delaware.


                            [SIGNATURE PAGES FOLLOW]


<PAGE>


         IN WITNESS WHEREOF, the parties hereto have caused this Fourth
Amendment to the Fourth Amended and Restated Registration Rights Agreement to be
executed by their respective officers thereunto duly authorized, as of the date
first above written.

                                            SPEECHWORKS INTERNATIONAL, INC.



                                            By:  /s/ STUART R. PATTERSON
                                               ---------------------------------
                                               Stuart Patterson, President and
                                               Chief Executive Officer


                                            NEW INVESTOR:

                                            NET2PHONE, INC.



                                            By: /s/ ILAN SLASKY
                                               ---------------------------------
                                                   Name: Ilan Slasky
                                                   Title: CFO


                                            INVESTORS:
                                            iGATE VENTURES I, L.P.



                                            By:  /s/ AJMAL NOURANI
                                               ---------------------------------
                                                   Name: Ajmal Nourani
                                                   Title: Partner


                                            REUTERS HOLDINGS SWITZERLAND SA
                                            [REUTERS INVESTMENTS (BERMUDA)
                                            LIMITED]



                                            By:  /s/ JEREMEY K
                                               ---------------------------------
                                                   Name: Jeremey K
                                                   Title: Director


<PAGE>


                                            CITICORP STRATEGIC TECHNOLOGY CORP.



                                            By:
                                               ---------------------------------
                                                     Name:
                                                     Title:


                                            GE CAPITAL EQUITY INVESTMENTS, INC.



                                            By:  /s/ ROBERT F. GOTTI
                                               ---------------------------------
                                                 Name: Robert F. Gotti
                                                 Title: Assistant Vice President


                                            MCI WORLDCOM VENTURE FUND, INC.



                                            By:
                                               ---------------------------------
                                                     Name:
                                                     Title:


                                           CHARLES RIVER PARTNERSHIP VII



                                            By:  /s/ RICHARD M. BURNES, JR.
                                               ---------------------------------
                                                    Name: Richard M. Burnes, Jr.
                                                    Title: General Partner


                                            ATLAS VENTURE FUND II, L.P.

                                            By:  Atlas Venture Associates II,
                                                 L.P., its general Partner


                                            By: /s/ JEAN FRANCOIS FORMELA
                                               ---------------------------------
                                                     Name: Jean Francois Formela
                                                     Title: General Partner


<PAGE>


                                            QUESTMARK PARTNERS, L.P.



                                            By:  /s/ THOMAS R. HITCHNER
                                               ---------------------------------
                                                     Name: Thomas R. Hitchner
                                                     Title: General Partner


                                            INTEL 64 FUND, LLC

                                            By:   INTEL 64 FUND OPERATIONS, INC.
                                                  its Coordinating Member



                                            By:
                                               ---------------------------------
                                                     Name:
                                                     Title:


                                                     Title:

                                            BANK OF AMERICA VENTURES


                                            By:  /s/ ROBERT OBUCH
                                               ---------------------------------
                                                     Name: Robert Obuch
                                                     Title: Principal


                                            BA VENTURE PARTNERS III

                                            By: /s/ ROBERT OBUCH
                                               ---------------------------------
                                                     Name: Robert Obuch
                                                     Title: General Partner


                                            BA VENTURE PARTNERS III

                                            By:  /s/ J. CARTER BEESE, JR.
                                               ---------------------------------
                                                     Name: J. Carter Beese, Jr.
                                                     Title: President


<PAGE>


                                         MINTZ LEVIN INVESTMENTS LLC

                                         By:
                                            ------------------------------------
                                                  Name:
                                                  Title:


                                            ------------------------------------
                                            Steven P. Rosenthal


                                            ------------------------------------
                                            Suzanne Abair and Kathleen MacDonald


                                            ------------------------------------
                                            Robert S. Fore


                                         LEE CAPITAL HOLDINGS


                                         By:
                                            ------------------------------------
                                                  Name:
                                                  Title:


                                         CITIZENS CAPITAL INCORPORATED


                                         By:
                                            ------------------------------------
                                                  Name:
                                                  Title:


                                         DIGITAL BANDWIDTH LLC


                                         By: /s/ DAVID B. WEINBERG
                                            ------------------------------------
                                                  Name: David B. Weinberg
                                                  Title: President


                                         INTEL CORPORATION


<PAGE>



                                         By:
                                            ------------------------------------
                                                  Name:
                                                  Title:


                                            SAP AMERICA, INC.


                                         By:  /s/ BRAD C. BRUBAKER
                                            ------------------------------------
                                                 Name: Brad C. Brubaker
                                                 Title: Sr. VP & General Counsel

                                              /s/ PAUL YOVOVICH
                                            ------------------------------------
                                            Paul Yovovich

                                            ------------------------------------
                                            Ralph Mor

                                            ------------------------------------
                                            Hedva Mor