<PAGE>

                                                                    Exhibit 10.1

                 FIFTH AMENDMENT TO FOURTH AMENDED AND RESTATED
                          REGISTRATION RIGHTS AGREEMENT

         THIS FIFTH AMENDMENT, dated as of January 5th, 2001 (this "Agreement"),
to the Fourth Amended and Restated Registration Rights Agreement dated as of
April 11, 2000 as amended by the First Amendment thereto dated May 16, 2000, the
Second Amendment thereto dated June 5, 2000, the Third Amendment thereto dated
June 28, 2000 and the Fourth Amendment thereto dated June 28, 2000 (the
"Registration Rights Agreement"), by and among SPEECHWORKS INTERNATIONAL, INC.,
a Delaware corporation (the "Company") and the stockholders of the Company named
therein, is by and among the Company, Dr. Susan Hertz (the "New Investor") and
the persons named on the signature page hereto as Investors (collectively, the
"Investors"). All other capitalized terms used herein and not otherwise defined
have their respective meanings set forth in the Registration Rights Agreement.

         WHEREAS, the Company and the New Investor are parties to the Agreement
and Plan of Merger, dated as of December 19, 2000 (the "Merger Agreement"), by
and among the Company, SpeechWorks Acquisition Sub, Inc. (the "Merger Sub"),
Eloquent Technology, Inc. ("Eloquent") and the New Investor pursuant to which
Eloquent will be merged with and into Merger Sub (the "Merger");

         WHEREAS, the New Investor is the sole shareholder of Eloquent;

         WHEREAS, upon consummation of the Merger and in consideration therefor,
the Company will issue 299,873 shares of the Common Stock, $0.001 par value per
share (the "Shares"), of the Company to the New Investor pursuant to the terms
of the Merger Agreement;

         WHEREAS, it is a condition to the consummation of the Merger that the
Registration Rights Agreement shall be amended to add the New Investor as a
party to the Registration Rights Agreement; and

         WHEREAS, the Investors who are signatories hereto hold in the aggregate
a sufficient number of shares of Registrable Stock to amend the Registration
Rights Agreement in accordance with Section 14 thereof to add the New Investor
as party thereto.

          NOW, THEREFORE, in consideration of the mutual covenants and
agreements set forth herein, the parties agree as follows:

         1. AMENDMENT TO SCHEDULE RRA (4TH REVISED). From and after the date
hereof, SCHEDULE RRA (4TH REVISED) to the Registration Rights Agreement is
deleted and replaced by SCHEDULE RRA (5TH REVISED), dated the date hereof and
attached hereto and the term "Investors" as used therein and herein shall mean
and refer to all the persons and entities identified on said SCHEDULE RRA (5TH
REVISED).

         2. CONTINUED EFFECT. As amended hereby, the Registration Rights
Agreement is hereby ratified and confirmed and agreed to by all of the parties
hereto and continues in full force and effect.

<PAGE>

         3. COUNTERPARTS. This Agreement may be executed in any number of
counterparts, each of which shall be deemed an original, but all of which
together shall constitute one and the same instrument.

         4. GOVERNING LAW. This Agreement shall be governed by, and construed in
accordance with the laws of the State of Delaware.

                            [signature pages follow]

                                       2

<PAGE>

         IN WITNESS WHEREOF, the parties hereto have caused this Fifth Amendment
to the Fourth Amended and Restated Registration Rights Agreement to be executed
by their respective officers thereunto duly authorized, as of the date first
above written.

                                     SPEECHWORKS INTERNATIONAL, INC.

                                     By: /s/ Stuart Patterson
                                     ------------------------------------------
                                     Stuart Patterson, President and Chief
                                     Executive Officer

                                     NEW INVESTOR:

                                     ------------------------------------------
                                     Dr. Susan Hertz

                                     INVESTORS:

                                     NET2PHONE, INC.

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                     AMERICA ONLINE, INC.

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                     AT&T CORP.

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                       3

Signature Page to Fifth Amendment to Fourth Amended and Restated Registration
Rights Agreement

<PAGE>

                                     IGATE VENTURES I, L.P.

                                     By: /s/ David C. Whitmore
                                        ---------------------------------------
                                     Name: David C. Whitmore
                                     Title: Manager

                                     REUTERS HOLDINGS SWITZERLAND SA

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                     CITICORP STRATEGIC TECHNOLOGY CORP.

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                     GE CAPITAL EQUITY INVESTMENTS, INC.

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                     MCI WORLDCOM VENTURE FUND, INC.

                                     By: /s/ Douglas C. Webster
                                        ---------------------------------------
                                     Name: Douglas C. Webster
                                     Title: Vice President

                                     CHARLES RIVER PARTNERSHIP VII, LP
                                       by Charles River VII GP, LP
                                          Its General Partner

                                     By: /s/ Richard M. Burnes
                                        ---------------------------------------
                                     Name: Richard M. Burnes
                                     Title: General Partner

                                       4

Signature Page to Fifth Amendment to Fourth Amended and Restated Registration
Rights Agreement

<PAGE>

                                     ATLAS VENTURE FUND II, L.P.

                                     By: Atlas Venture Associates II, L.P., its
                                     general partner

                                     By: /s/ Barry J. Fidelman
                                        ---------------------------------------
                                     Name: Barry J. Fidelman
                                     Title: General Partner

                                     QUESTMARK PARTNERS, L.P.

                                     By: /s/ Thomas R. Hitchner
                                        ---------------------------------------
                                     Name: Thomas R. Hitchner
                                     Title: General Partner

                                     INTEL 64 FUND, LLC
                                     By: INTEL 64 FUND OPERATIONS, INC.,
                                     its Coordinating Member

                                     By: /s/ Kala Srinivasan
                                        ---------------------------------------
                                     Name: Kala Srinivasan
                                     Title: Assistant Treasurer

                                     BANK OF AMERICA VENTURES

                                     By: /s/ Robert M. Obuch
                                        ---------------------------------------
                                     Name: Robert M. Obuch
                                     Title: Principal

                                     BA VENTURE PARTNERS III

                                     By: /s/ Robert M. Obuch
                                        ---------------------------------------
                                     Name: Robert M. Obuch
                                     Title: General Partner

                                       5

Signature Page to Fifth Amendment to Fourth Amended and Restated Registration
Rights Agreement

<PAGE>

                                     RIGGS CAPITAL PARTNERS

                                     By: /s/ J. Carter Beese, Jr.
                                        ---------------------------------------
                                     Name: J. Carter Beese, Jr.
                                     Title: President

                                     MINTZ LEVIN INVESTMENTS LLC

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                     ------------------------------------------
                                     Steven P. Rosenthal

                                     ------------------------------------------
                                     Suzanne Abair and Kathleen MacDonald

                                     /s/ Robert S. Fore
                                     ------------------------------------------
                                     Robert S. Fore

                                     LEE CAPITAL HOLDINGS

                                     By: /s/ O. Lee
                                        ---------------------------------------
                                     Name: O. Lee
                                     Title: President

                                     CITIZENS CAPITAL INCORPORATED

                                     By: /s/ Robert E. Garrow
                                        ---------------------------------------
                                     Name: Robert E. Garrow
                                     Title:President

                                     /s/ Joseph Murphy
                                     ------------------------------------------
                                     JOSEPH MURPHY

                                       6

Signature Page to Fifth Amendment to Fourth Amended and Restated Registration
Rights Agreement

<PAGE>
                                     DIGITAL BANDWIDTH LLC

                                     By:
                                        ---------------------------------------
                                     Name: David B. Weinberg
                                     Title: President

                                     INTEL CORPORATION

                                     By: /s/ Kala Srinivasan
                                        ---------------------------------------
                                     Name: Kala Srinivasan
                                     Title: Assistant Treasurer

                                     SAP AMERICA, INC.

                                     By:
                                        ---------------------------------------
                                     Name:
                                     Title:

                                     /s/ Paul Yovovich
                                     ------------------------------------------
                                     Paul Yovovich

                                     /s/ Ralph Mor
                                     ------------------------------------------
                                     Ralph Mor

                                     ------------------------------------------
                                     Hedva Mor

                                     ------------------------------------------
                                     Jeffrey Mor

                                     /s/ Jean Guy Dahan
                                     ------------------------------------------
                                     Jean Guy Dahan

                                     /s/ Naim Murad
                                     ------------------------------------------
                                     Naim Murad

                                     /s/ John Meyrick
                                     ------------------------------------------
                                     John Meyrick

                                     /s/ Brett Phaneuf
                                     ------------------------------------------
                                     Brett Phaneuf

                                       7

Signature Page to Fifth Amendment to Fourth Amended and Restated Registration
Rights Agreement

<PAGE>

                                     ------------------------------------------
                                     William J. O'Farrell

                                     ------------------------------------------
                                     Noreen D. O'Farrell

                                     ------------------------------------------
                                     William Ledingham

                                     /s/ William Haney
                                     ------------------------------------------
                                     William Haney

                                     /s/ Anne G. Haney
                                     ------------------------------------------
                                     Anne G. Haney

                                     LIGHTHOUSE CAPITAL PARTNERS, L.P.

                                     By: LIGHTHOUSE MANAGEMENT
                                         PARTNERS, L.P., its general partner

                                     By: LIGHTHOUSE CAPITAL
                                         PARTNERS, INC., its general partner

                                     By:
                                        ---------------------------------------
                                     Title:
                                            -----------------------------------

                                     /s/ Mark Holthouse
                                     ------------------------------------------
                                     Mark Holthouse

                                     /s/ Stephen Smith
                                     ------------------------------------------
                                     Stephen Smith

                                     /s/ Sol Lerner
                                     ------------------------------------------
                                     Sol Lerner

                                     ------------------------------------------
                                     Henry Lerner

                                     /s/ Bernice F. Epstein, Trustee of
                                         Leonard Epstein Revocable Trust
                                     ------------------------------------------
                                     Leonard Epstein Revocable Trust

                                       8

Signature Page to Fifth Amendment to Fourth Amended and Restated Registration
Rights Agreement

<PAGE>

                                     ------------------------------------------
                                     Bella Lerner

                                     ------------------------------------------
                                     Miriam Epstein

                                     /s/ Brian S. Eberman
                                     ------------------------------------------
                                     Brian S. Eberman

                                     /s/ Edward J. McCaffrey
                                     ------------------------------------------
                                     Edward J. McCaffrey

                                     /s/ Joyce V. McCaffrey
                                     ------------------------------------------
                                     Joyce V. McCaffrey

                                     INTERVOICE-BRITE, INC.

                                     By: /s/ Rob Roy J. Graham
                                        ---------------------------------------
                                     Name: Rob Roy J. Graham
                                     Title: CFO


                                       9

Signature Page to Fifth Amendment to Fourth Amended and Restated Registration
Rights Agreement

<PAGE>

                           SCHEDULE RRA (5TH REVISED)
                                 January 5, 2001

INVESTORS AND MAILING ADDRESSES:
-------------------------------

Dr. Susan Hertz
24 Highgate Circle
Ithaca, NY  14850

Net2Phone, Inc.
171 Main Street
Hackensack, NJ  07601
Attn: General Counsel
Fax: (201) 530-4159

America Online, Inc.
22000 AOL Way
Dulles, Virginia  20166
Attn: General Counsel

AT&T Corp.
295 North Maple Avenue
Basking Ridge, NJ  07920

iGate Ventures I, L.P.
1004 McKee Road
Oakdale, PA 15071

Reuters Holdings Switzerland SA
153 Route de Thonon
1245 Collogne-Bellerive
Switzerland
(with copies to: Reuters Limited
85 Fleet Street
London EC4P 4AJ
United Kingdom
Attention: General Counsel)

Citicorp Strategic Technology Corp.
909 Third Avenue,
16th Floor, New York, NY 10043
Attention: William F. Carson

                                       10

<PAGE>

GE Capital Equity Investments, Inc.
120 Long Ridge Road, Stamford, CT 06927
Attention: General Counsel

MCI WorldCom Venture Fund, Inc.
1801 Pennsylvania Avenue N.W.
6th Floor
Washington D.C. 20006

Mintz Levin Investments LLC
One Financial Center
Boston, MA  02111
Attention: Steven P. Rosenthal, Esq.

Steven P. Rosenthal
40 Bartlett Street
Marblehead, MA  01945

Suzanne Abair & Kathleen MacDonald
130 Pembroke St. #1
Boston, MA 02118

Atlas Venture Fund II, L.P.
222 Berkeley Street
Boston, MA 02116
Attention: Axel Bichara

Charles River Partnership VII
1000 Winter Street
Suite 3300
Waltham, MA 02154
Attention:  Richard Burnes

Digital Bandwidth LLC
Bank One Plaza
21 South Clark Street, Suite 3140
Chicago, IL  60603
Attention: David B. Weinberg

                                       11

<PAGE>

Bank of America Ventures
950 Tower Lane, Suite 700
Foster City, CA  94404
Attention: Robert Obuch

BA Venture Partners III
950 Tower Lane, Suite 700
Foster City, CA  94404
Attention: Robert Obuch

Intel Corporation
5200 N.E. Elam Young Parkway
Hillsboro, OR  07123
Attention: Ken Matthews

SAP America, Inc.
3999 WestChester Pike
Newton Square,  PA 19073
Attention: Gary Fromer

QuestMark Partners, L.P.
One South Street, Suite 800
Baltimore, Maryland  21202
Attention: Tim Krongard

Riggs Capital Partners
800 17th Street, N.W.
Washington, DC  20006-3944
Attention: Vicken  Dombalagian

Intel 64 Fund Operations, Inc.
2200 Mission College Blvd.
Santa Clara, CA  95052
Attn: Portfolio Manager - M/S: RN6-46
With a copy to:
Intel Corporation
2200 Mission College Blvd.
Santa Clara, CA  95052
Attention:  General Counsel

Robert S. Fore
520 Georgetown Avenue
San Mateo, CA  94402

Edward J. and Joyce V. McCaffrey
23 Warwick Road
Winnetka, IL 60093

                                       12

<PAGE>

Paul Yovovich
Lake Capital
676 N. Michigan, Suite 3900
Chicago, IL 60611

Ralph Mor
11 Brook Road
Sharon, MA  02067

Hedva Mor
Rechov Hatziporen
6-Aleph
Bet Shemesh, ISRAEL 99000

Jeffrey Mor
Rechov Hatziporen
6-Aleph
Bet Shemesh, ISRAEL 99000

Jean Guy Dahan
12 Colburne Crescent, Apt. #3
Brookline, MA 02146

Naim Murad
75 Glengarry #502
Town of Mount Royal, PQ
H3R 1A2
Canada

John Meyrick
22 Lotus Avenue
Scituate, MA 02066

Brett Phaneuf
380 Pine Street
Marshfield, MA 02050

William J. O'Farrell
76 Taber Avenue
Providence, RI 02906

Noreen D. O'Farrell
76 Taber Avenue
Providence, RI  02906

William Ledingham
15 Wingate Road
Wellesley, MA 02181

                                       13

<PAGE>

William Haney
61 Lincoln Road
Wayland, MA 01778

Anne G. Haney
61 Lincoln Road
Wayland, MA 01778

Lighthouse Capital Partners, L.P.
500 Drake's Landing, Suite 260
Greenbrae, CA 94904-3121

Mark Holthouse
163 Upland Road
Newtonville, MA 02460-2420

Stephen Smith
4806 Jamestown Road
Bethesda, MD 20816

Sol Lerner
10 Flintlock Road
Sharon, MA 02067

Henry Lerner
One Celler Road
Edison, NJ

Leonard Epstein
193 Cheswick Road
Brighton, MA 02135

Bella Lerner
One Celler Road
Edison, NJ

Miriam Epstein
193 Chiswick Road
Brighton, MA  02135

Brian S. Eberman
10 LaFayette Road
Newton, MA 02162

                                       14

<PAGE>

Lee Capital Holdings, LLC
One International Place
Boston, MA 02110
Attn: Jonathan Lee

Citizens Capital Incorporated
28 State Street, 15th Floor
Boston, MA  02109
Attn: Robert Garrow

Joseph Murphy
40 Maynard Farm Road
Sudbury, MA  01776

Intervoice-Brite, Inc.
17811 Waterview Parkway
Dallas, TX 75252

                                       15