<PAGE>

                                                                     EXHIBIT 4.2

                                                               EXECUTION VERSION

                             STOCKHOLDERS AGREEMENT

      This STOCKHOLDERS AGREEMENT (the "AGREEMENT") is made as of March 19, 2004
by and between SCANSOFT, Inc., a corporation organized and existing under the
laws of the State of Delaware (the "COMPANY"), and Warburg Pincus Private Equity
VIII, L.P., Warburg Pincus Netherlands Private Equity VIII I C.V., Warburg
Pincus Netherlands Private Equity VIII II C.V., and Warburg Pincus Germany
Private Equity VIII K.G. (collectively, the "PURCHASERS").

                                    RECITALS

      A.    The Purchasers are acquiring Beneficial Ownership of an aggregate of
15,941,572 shares of the Voting Stock of the Company from Xerox Imaging Systems,
Inc. (the "TRANSACTION");

      B.    On March 15, 2004 the Company's Board of Directors approved and
granted to the Purchasers a series of warrants which, as of the date hereof and
subject to the provisions thereof, allow for the purchase of up to an aggregate
of 2,500,000 shares of Company Common Stock (the "WARRANTS").

      C.    The Purchasers have made certain requests, including (i) the
appointment of one (1) representative to the Board of Directors of the Company
(the "BOARD") and (ii) approval, for purposes of Section 203 of the Delaware
General Corporation Law (including any successor statute thereto "SECTION 203")
of the transactions pursuant to which one or more of the Purchasers will become,
together with their Affiliates, an "interested stockholder" within the meaning
of Section 203.

      D.    The Board has determined that it is in the best interests of the
stockholders of the Company to (i) issue the Warrants, (ii) appoint one (1)
designee of the Purchasers to the Board, and (iii) approve, for purposes of
Section 203, the transactions pursuant to which one or more of the Purchasers
will become, together with their Affiliates, an "interested stockholder" within
the meaning of Section 203.

      E.    The Company and the Purchasers desire to make certain covenants and
agreements with one another pursuant to this Agreement.

      NOW THEREFORE, in consideration of the covenants and promises set forth
herein, and for other good and valuable consideration, the receipt and
sufficiency of which are hereby acknowledged, the parties hereby agree as
follows:

                                   AGREEMENT

      1.    Certain Definitions. Unless the context otherwise requires, the
following terms, for all purposes of this Agreement, shall have the meanings
specified in this Section 1:

<PAGE>

      "AFFILIATE" shall have the meaning set forth in Rule 12b-2 of the rules
and regulations promulgated under the Exchange Act; provided, however, that for
purposes of this Agreement, the Purchasers and their Affiliates, on the one
hand, and the Company and its Affiliates, on the other, shall not be deemed to
be "Affiliates" of one another.

      "BENEFICIALLY OWN," "BENEFICIALLY OWNED," or "BENEFICIAL OWNERSHIP" shall
have the meaning set forth in Rule 13d-3 of the rules and regulations
promulgated under the Exchange Act.

      "BOARD APPOINTMENT PERIOD" shall mean the period beginning on the
Effective Date and ending on the date that the Purchasers cease to Beneficially
Own at least 10,000,000 shares of Voting Stock (as adjusted from time to time
for any stock dividends, combinations, splits, recapitalizations and the like).

      "CHANGE IN CONTROL OF THE COMPANY" shall mean any of the following: (i) a
merger, consolidation or other business combination or transaction to which the
Company is a party if the stockholders of the Company immediately prior to the
effective date of such merger, consolidation or other business combination or
transaction, as a result of such share ownership, have Beneficial Ownership of
voting securities representing less than 50% of the Voting Power of the
surviving entity following such merger, consolidation or other business
combination or transaction; (ii) an acquisition by any person, entity or 13D
Group of direct or indirect Beneficial Ownership of Voting Stock of the Company
representing 50% or more of the Voting Power of the Company; or (iii) a sale of
all or substantially all of the assets of the Company.

      "COMPANY COMMON STOCK" shall mean shares of the Common Stock of the
Company, $0.001 par value.

      "COMPANY COMPETITOR" shall mean any person or entity (or any Affiliates of
such a person or entity) that (i) conducts material activities and operations
consisting of providing speech technology primarily for use in telephony-network
based services, mobile or embedded platforms, or desktop or server-based
dictation software applications, (ii) develops and licenses software that
incorporates document capture or image processing technology, (iii) has filed a
statement on Schedule 13D pursuant to Rule 13d-1(a) with the SEC that indicates
under Item 4 of such Schedule that the person has acquired or holds the
securities with a purpose or effect of changing or influencing control of the
Company, or in connection with or as a participant in any transaction having
that purpose or effect, or (iv) to the Purchasers' knowledge after a written
request, intends to file a statement on Schedule 13D with the SEC indicating
under Item 4 of such Schedule that the person has acquired or holds the
securities with a purpose or effect of changing or influencing control of the
Company, or in connection with or as a participant in any transaction having
that purpose or effect.

      "EFFECTIVE DATE" shall mean the date on which the Warrants shall first
become exercisable in accordance with their terms.

      "EXCHANGE ACT" shall mean the Securities Exchange Act of 1934, as amended.

                                       2
<PAGE>

      "EXCHANGE OFFER" shall mean a bona fide exchange offer subject to the
provisions of Rule 13e-3 promulgated under the Exchange Act.

      "FAIR MARKET VALUE" means, as of any date of determination, (i) in the
case of Company Common Stock, the average of the closing sale prices of Company
Common Stock during the 5 trading days immediately preceding such date of
determination on the principal U.S. or foreign securities exchange on which such
Company Common Stock is listed or, if such Company Common Stock is not listed or
primarily traded on any such exchange, the average of the closing sale prices or
the closing bid quotations of such security during the 5 day period preceding
such date of determination on Nasdaq or any comparable system then in use or, if
no such quotations are available, the fair market value of such security as of
such date of determination as determined in good faith by the Company and the
holders of a majority in interest of the shares of Company Common Stock then
held by the Purchasers and (ii) in the case of property other than cash or a
security, the fair market value of such property on such date of determination
as determined in good faith by the Company and the holders of a majority in
interest of the shares of Company Common Stock then held by the Purchasers;
provided, however, that if such parties are unable to reach agreement as to the
fair market value of such security pursuant to clause (i) above or such property
pursuant to clause (ii) above within a reasonable period of time, the fair
market value shall be determined in good faith by an independent investment
banking firm selected jointly by the Company and the holders of a majority in
interest of the shares of Company Common Stock then held by the Purchasers or,
if that selection cannot be made within 15 days, by an independent investment
banking firm selected by the American Arbitration Association in accordance with
its rules. All costs and expenses of such independent investment banking firm
shall be borne 50% by the Company and 50% by the Purchasers, pro rata based on
the number of shares of Company Common Stock then held by each.

      "FORM S-3" means such form under the Securities Act as in effect on the
date hereof or any successor or similar registration form under the Securities
Act subsequently adopted by the SEC that permits inclusion or incorporation of
substantial information by reference to other documents filed by the Company
with the SEC.

      "HOLDER" means any person owning of record Registrable Securities that
have not been sold to the public or any transferee or assignee of record of such
Registrable Securities to which the registration rights conferred by this
Agreement have been transferred or assigned in accordance with Section 5.8
hereof.

      "LOCK-UP PERIOD" shall mean the period beginning on the Effective Date and
ending at 12:00 a.m. New York City time on the 365th day after the Effective
Date.

      "REGISTER," "REGISTERED" and "REGISTRATION" refer to a registration
effected by preparing and filing a registration statement in compliance with the
Securities Act, and the declaration or ordering of effectiveness of such
registration statement or document.

      "REGISTRABLE SECURITIES" means (a) shares of Company Common Stock (i) held
by the Purchasers on the day immediately following the Effective Date or (ii)
acquired pursuant to open

                                       3
<PAGE>

market purchases following the Effective Date, and (b) any Company Common Stock
issued as (or issuable upon the conversion or exercise of any warrant, right or
other security which is issued as) a dividend or other distribution with respect
to, or in exchange for or in replacement of, any Company Common Stock, warrant,
right or other security held by the Purchasers. Notwithstanding the foregoing,
Registrable Securities shall not include any securities of the Company sold by
any person to the public either pursuant to a registration statement under the
Securities Act or Rule 144.

      "REGISTRATION EXPENSES" shall mean all expenses incurred by the Company in
complying with Sections 5.1 and 5.2 hereof, including, without limitation, all
registration and filing fees, printing expenses, fees and disbursements of
counsel for the Company, Blue Sky fees and expenses and the expense of any
special audits incidental to or required by any such registration (but excluding
the compensation of regular employees of the Company, which shall be paid in any
event by the Company, and all underwriting discounts and commissions).

      "RULE 144" means Rule 144 as promulgated by the SEC under the Securities
Act, as such rule may be amended from time to time, or any similar successor
rule that may be promulgated by the SEC.

      "SEC" or "COMMISSION" means the Securities and Exchange Commission or any
other federal agency at the time administering the Securities Act.

      "SECURITIES ACT" means the Securities Act of 1933, as amended, or any
similar successor federal statute and the rules and regulations thereunder, all
as the same shall be in effect from time to time.

      "SELLING EXPENSES" means all underwriting discounts, selling commissions
and stock transfer rates applicable to the sale of Registrable Securities and,
except as set forth in the definition of "Registration Expenses" above, all fees
and reimbursement of counsel for the Holders.

      "SHELF REGISTRATION PERIOD" shall mean the period beginning on the 548th
day after the Effective Date and ending at 12:00 a.m. New York City time on the
365th day after the effectiveness of any registration statement filed pursuant
to the terms of Section 5.3, as such period may be extended.

      "THIRD PARTY TENDER OFFER" shall mean a bona fide public tender offer
subject to the provisions of Regulation 14D when first commenced within the
meaning of Rule 14d-2(a) of the rules and regulations under the Exchange Act, by
a person or 13D Group (which is not made by and does not include any of the
Company or any Affiliate of the Company) to purchase or exchange for cash or
other consideration any Voting Stock and which consists of an offer to acquire
more than 10% of the Voting Power of the Company.

      "13D GROUP" means any group of persons formed for the purpose of
acquiring, holding, voting or disposing of Voting Stock which would be required
under Section 13(d) of the Exchange Act, and the rules and regulations
promulgated thereunder, to file a statement on Schedule 13D pursuant to Rule
13d-1(a) or Schedule 13G pursuant to Rule 13d-1(c) with the SEC as a "person"

                                       4
<PAGE>

within the meaning of Section 13(d)(3) of the Exchange Act if such group
Beneficially Owned Voting Stock representing more than 5% of any class of Voting
Stock then outstanding.

      "VOTING POWER" shall mean the number of votes entitled to then be cast by
the Voting Stock of the Company at any election of directors of the Company,
provided that, for the purpose of determining Voting Power, each share of
Preferred Stock of the Company, if any (the "PREFERRED STOCK"), shall be deemed
to be entitled to the number of votes equal to the number of shares of Company
Common Stock into which such share of Preferred Stock could then be converted.

      "VOTING STOCK" shall mean shares of the Company Common Stock and any other
securities of the Company having the ordinary power to vote in the election of
members of the Board of Directors of the Company and any securities convertible,
exchangeable for or otherwise exercisable to acquire voting securities.

      2.    Appointment of Purchasers' Nominee to the Board. The Company hereby
agrees that not later than two (2) business days following the Effective Date
and until the expiration of the Board Appointment Period, the Board shall take
such action as may be necessary to appoint one (1) member of the Board who shall
be designated by the holders of a majority in interest of the shares of Company
Common Stock then held by the Purchasers, and shall be reasonably acceptable to,
and approved by a majority of the Board, which acceptance and approval shall not
be unreasonably withheld (such nominee from time to time so designated, the
"PURCHASER NOMINEE"), which Purchaser Nominee shall initially be William
Janeway. The Company shall, throughout the Board Appointment Period, nominate
and take such action as may be necessary to cause the Purchaser Nominee to be
elected or appointed to the Board. If at any time during the Board Appointment
Period there shall occur a vacancy in the Board seat previously occupied by a
Purchaser Nominee by reason of resignation, removal, death or incapacity, then
such vacancy shall be filled by another Purchaser Nominee designated in
accordance with this Section 2.

      3.    Covenants of the Purchasers.

            3.1   Lock-up Restrictions. For the Lock-Up Period, the Purchasers
hereby agree not to, directly or indirectly, sell, transfer, pledge, contract to
sell, sell any option or contract to purchase, purchase any option or contract
to sell, grant any option, right or warrant to purchase, transfer the economic
risk of ownership of, or otherwise dispose of (each, a "TRANSFER") any Voting
Stock or Voting Power to any person (other than an Affiliate of the Purchasers
who agrees in writing to be bound by the terms of this Agreement and which does
not have the effect of increasing or decreasing the aggregate amount of Voting
Stock or Voting Power held by the Purchasers and their Affiliates), except in
connection with the closing of a Change in Control of the Company that has been
approved by a majority of the Board.

            3.2   Transfer Restrictions. The Purchasers shall not (and shall not
permit any Affiliate to), directly or indirectly:

                  (a)   Transfer five percent (5%) or more of the Voting Stock
or Voting Power (in one or a series of transactions) in response to a Third
Party Tender Offer or an Exchange

                                       5
<PAGE>

Offer with respect to which the Board shall not have recommended that
stockholders of the Company accept such offer, unless prior to such Transfer,
the Purchasers have complied with the provisions of Section 3.3 below; or

                  (b)   Transfer five percent (5%) or more of the Voting Stock
or Voting Power (in one or a series of transactions that have not been approved
by a majority of the Board) to a Company Competitor who has made a bona fide
written offer to acquire such securities, unless prior to such Transfer, the
Purchasers have complied with the provisions of Section 3.3 below.

            3.3   The Company's Right of First Refusal.

                  (a)   Prior to the Purchasers effecting any Transfer of Voting
Stock or Voting Power that is subject to the restrictions set forth in Section
3.1 and Section 3.2, the Company shall have a first refusal right to purchase
such Voting Stock or Voting Power on the following terms and conditions:

                        (i)   The Purchasers shall give prior notice (the
"TRANSFER NOTICE") to the Company in writing of such intention, specifying the
name of the proposed purchaser or transferee, the amount of Voting Stock or
Voting Power proposed to be the subject of such Transfer, the proposed price
therefor and the other material terms upon which such disposition is proposed to
be made (including, if any, a copy of a bona fide written offer).

                        (ii)  The Company shall have the right, exercisable by
written notice given by the Company to the Purchasers within (i) 72-hours with
respect to a Transfer addressed in Section 3.2(a) above, and (ii) twenty (20)
business days with respect to a Transfer addressed in Section 3.2(b) above,
after receipt of such Transfer Notice (the "RESPONSE NOTICE"), to purchase all
or any portion of the Voting Stock or Voting Power specified in such Transfer
Notice for cash at the price per share specified in the Transfer Notice or, if
consideration other than cash is specified in the Transfer Notice, in an amount
equal to the Fair Market Value of such non-cash consideration.

                        (iii) If the Company exercises its right of first
refusal hereunder, the closing of the purchase of the Voting Stock or Voting
Power with respect to which such right has been exercised shall take place
within thirty (30) calendar days after the Company gives the Response Notice to
the Purchasers or, if later, within five (5) business days of the determination
of the Fair Market Value of any non-cash consideration. Upon exercise of its
right of first refusal, the Company and the Purchasers shall be legally
obligated to consummate the purchase and sale contemplated thereby and shall use
their commercially reasonable efforts to secure any approvals required in
connection therewith.

                        (iv)  If the Company does not exercise its right of
first refusal hereunder within the time specified for such exercise in
subparagraph (ii) above with respect to all of the Voting Stock or Voting Power
specified in such Transfer Notice, the Purchasers shall be free, during the
period of ninety (90) calendar days following the expiration of such time for
exercise, to Transfer or tender for Transfer the Voting Stock or Voting Power
specified in such Transfer Notice

                                       6
<PAGE>
with respect to which the Company has not exercised its first refusal rights to
the proposed purchaser or transferee specified in such Transfer Notice and on
terms not materially less favorable to the Purchasers than the terms specified
in such Transfer Notice. After the expiration of such 90-day period, except as
otherwise provided herein, the Purchasers may not Transfer the Voting Stock or
Voting Power specified in such Transfer Notice without first complying with the
provisions of this Section 3.3.

                  (b)   The Company may assign its right of first refusal under
this Section 3.3 to any other person or persons; provided, however, that the
Company shall be liable for the timely performance of any obligations in this
Section 3.3 by such assignee.

            3.4   Other Transfers. Except as provided in this Section 3, this
Agreement does not, and is not intended to, restrict the Purchasers ability to
Transfer any Voting Stock or Voting Power.

      4.    Prohibited Transfer. Any purchase which causes the Purchasers to be
in violation of the terms of Section 3 above ("PROHIBITED TRANSFER") shall not
be effected by the Company and shall be voidable at the option of the Company by
their giving written notice to the transferor, his transferee and the
Purchasers. Each certificate representing Voting Stock held by the Purchasers
shall be endorsed by the Company with a legend reading as follows:

      "THE SHARES EVIDENCED HEREBY ARE SUBJECT TO A STOCKHOLDERS AGREEMENT BY
      AND BETWEEN THE COMPANY AND THE HOLDER HEREOF (A COPY OF WHICH MAY BE
      OBTAINED FROM THE COMPANY), AND NO TRANSFER OF THE SHARES EVIDENCED HEREBY
      SHALL BE EFFECTIVE EXCEPT IN COMPLIANCE WITH THE TERMS THEREOF."

      5.    Registration Rights.

            5.1   Demand Registration.

                  (a)   Subject to the conditions of this Section 5.1, if the
Company shall receive a written request from the Holders holding not less than a
majority of the Registrable Securities then outstanding that the Company file a
registration statement with respect to all or part of the Registrable Securities
under the Securities Act with an anticipated aggregate offering price of at
least $10,000,000, then the Company shall, within ten (10) calendar days of the
receipt thereof, give written notice of such request to all Holders, and,
subject to the limitations of this Section 5.1, use its commercially reasonable
efforts to effect, as expeditiously as reasonably possible, the registration
under the Securities Act of all Registrable Securities that all Holders request
to be registered pursuant to and in accordance with this Agreement (a "DEMAND
REGISTRATION").

                  (b)   Notwithstanding the foregoing, if the Company shall
furnish to the Holders requesting a registration statement pursuant to this
Section 5.1, a certificate signed by the President or Chief Executive Officer of
the Company stating that in the Board's good faith judgment it would be
seriously detrimental to the Company and its stockholders for such a
registration

                                       7
<PAGE>

statement to be filed in the near future, the Company shall have the right to
defer such filing for a period of not more than ninety (90) days after receipt
of the request of the Holders specified in Section 5.1(a); provided, however,
that the Company may not utilize this right more than twice in any twelve-month
period.

                  (c)   The Company shall not be required to effect or take any
action to effect a registration pursuant to this Section 5.1:

                        (i)   prior to the 548th day after the Effective Date;

                        (ii)  after the Company has effected two Demand
Registrations pursuant to this Section 5.1, and such registrations have been
declared or ordered effective (which, for the avoidance of doubt, shall mean
that the registrations shall have been effective for an aggregate of ninety (90)
calendar days, or until all Registrable Securities covered thereby have been
sold, if earlier);

                        (iii) if the Holders making the request provided for in
Section 5.1(a) propose to dispose of Registrable Securities that could be
disposed of in a single ordinary brokerage transaction under the quantity
limitation of Rule 144; or

                        (iv)  if the Holders making the request provided for in
Section 5.1(a) propose to dispose of Registrable Securities that may be
registered on Form S-3 pursuant to a request made pursuant to Section 5.2 below.

            5.2   Form S-3 Registration. If at any time following the 548th day
following the Effective Date, the Company shall receive from the Holders holding
not less than a majority of the Registrable Securities then outstanding a
written request that the Company effect a registration on Form S-3 and any
related qualification or compliance with respect to all or a part of the
Registrable Securities owned by such Holders, the Company will:

                  (a)   within ten (10) calendar days after receipt of such
notice, give written notice of the proposed registration, and any related
qualification or compliance, to all other Holders of Registrable Securities; and

                  (b)   as soon as reasonably practicable, effect such
registration (a "S-3 REGISTRATION") and all such qualifications and compliances
as may be so requested and as would permit or facilitate the sale and
distribution of all or such portion of such Holders' Registrable Securities as
are specified in such request, together with all or such portion of the
Registrable Securities of any other Holder or Holders joining in such request as
are specified in a written request given within fifteen (15) calendar days after
receipt of such written notice from the Company; provided, however, that the
Company shall not be obligated to effect any such registration, qualification or
compliance pursuant to this Section 5.2, (i) if Form S-3 is not available to the
Company for such offering, (ii) if the aggregate proceeds from the sale of
Registrable Securities proposed to be sold pursuant to a Form S-3 registration
statement will not exceed $10,000,000, (iii) if, the Company has effected two
S-3 Registrations pursuant to this Section 5.2, and such

                                       8
<PAGE>

registrations have been declared or ordered effective (which, for the avoidance
of doubt, shall mean that the registrations shall have been effective for an
aggregate of ninety (90) calendar days, or until all Registrable Securities
covered thereby have been sold, if earlier), or (iv) if the Holders propose to
dispose of Registrable Securities that could be disposed of in a single ordinary
brokerage transaction under the quantity limitation of Rule 144.

                  (c)   Notwithstanding the foregoing, if the Company shall
furnish to the Holders requesting a registration statement pursuant to this
Section 5.2, a certificate signed by the President or Chief Executive Officer of
the Company stating that in the Board's good faith judgment it would be
seriously detrimental to the Company and its stockholders for such a
registration statement to be filed in the near future, the Company shall have
the right to defer such filing for a period of not more than ninety (90) days
after receipt of the request of the Holders specified in this Section 5.2;
provided, however, that the Company may not utilize this right more than twice
in any twelve-month period.

      Registrations effected pursuant to this Section 5.2 shall be counted as
demands for registration effected pursuant to Section 5.1, and in no event shall
the Company be required to effect more than two (2) S-3 Registrations.

            5.3   Shelf Registration. During the Shelf Registration Period, if
the Company shall receive from the Holders holding not less than a majority of
the Registrable Securities then outstanding a written request that the Company
effect a registration on Form S-3 with respect to all or part of the Registrable
Securities owned by such Purchasers, the Company will as soon as reasonably
practicable, effect such registration (a "SHELF REGISTRATION STATEMENT") and all
such qualifications and compliances as may be so requested and as would permit
or facilitate the distribution of all or such portion of such Holders'
Registrable Securities as are specified in such request exclusively to partners,
limited partners, retired partners, retired limited partners, members, retired
members and stockholders of such Holders; provided, however, that the Company
shall not be obligated to effect any such registration, qualification or
compliance pursuant to this Section 5.3, if Form S-3 is not available to the
Company for such offering. The Company shall use its commercially reasonable
efforts to keep the Shelf Registration Statement effective, current and
available for use by the Purchasers during the Shelf Registration Period. While
the Shelf Registration Statement remains in effect, the Company may at any time
and from time to time deliver to the Holders written notice to the effect that
distributions may not be effected under the Shelf Registration Statement for a
period of time (a "BLACKOUT PERIOD") because of the existence of material facts
not disclosed or incorporated by reference in such Shelf Registration Statement
and in the then-current prospectus included therein; provided, however, that the
duration of any Blackout Period shall not exceed ninety (90) days. Upon receipt
of any such notice, the Holders shall refrain from distributing Registrable
Securities under such Shelf Registration Statement until the Holders have
received notice from the Company to the effect that such distributions may then
be effected. The Company shall as promptly as reasonably possible update the
Shelf Registration Statement and the prospectus included therein in order to
permit Registrable Securities to be distributed, and the Shelf Registration
Period shall automatically be extended by the aggregate number of days during

                                       9
<PAGE>

which the Holders were instructed to refrain from distributing Registrable
Securities during all Blackout Periods, without duplication.

            5.4   Expenses of Registration. Except as specifically provided
herein, all Registration Expenses incurred in connection with any registration
effected pursuant to Section 5.1, Section 5.2 or Section 5.3 herein shall be
borne by the Company. All Selling Expenses incurred in connection with any
registrations effected pursuant to Section 5.1 or Section 5.2 shall be borne by
the holders of the securities so registered pro rata on the basis of the number
of shares so registered. The Company shall not, however, be required to pay for
expenses of any registration proceeding begun the request of which has been
subsequently withdrawn by the Holders initiating such registration unless a
majority of the Holders of Registrable Securities agree to forfeit their right
to one Demand Registration or S-3 Registration, as the case may be, pursuant to
Section 5.1 or Section 5.2 (in which event such right shall be forfeited by all
Holders of Registrable Securities). If such Holders are required to pay the
Registration Expenses, such expenses shall be borne by the holders of securities
(including Registrable Securities) initiating such registration in proportion to
the number of shares for which registration was requested.

                                      10
<PAGE>

            5.5   Underwriting. If the Company determines in its sole discretion
that the registration statement under which the Company gives notice under
Section 5.1 or Section 5.2 will be for an underwritten offering, the Company
shall so advise the Holders of Registrable Securities. In such event, the right
of any such Holder to be included in such registration shall be conditioned upon
such Holder's participation in such underwriting and the inclusion of such
Holder's Registrable Securities in the underwriting to the extent provided
herein. All Holders proposing to distribute their Registrable Securities through
such underwriting shall enter into an underwriting agreement in customary form
with the underwriter or underwriters selected for such underwriting by the
Company. Notwithstanding any other provision contained in this Agreement, if the
managing underwriter determines in good faith that marketing factors require a
limitation of the number of shares to be underwritten (including Registrable
Securities), the number of shares that may be included in the underwriting shall
be allocated first to requesting Holders on a pro rata basis based on the total
number of Registrable Securities then held by all such requesting Holders, then
to any other shareholders on a pro rata basis based on the number of shares of
Company Common Stock then held by such other shareholders. If any Holder
disapproves of the terms of any such underwriting, such Holder may elect to
withdraw therefrom by written notice to the Company and the managing
underwriter, delivered at least ten (10) business days prior to the effective
date of the registration statement. Any Registrable Securities excluded or
withdrawn from such underwriting shall be excluded and withdrawn from the
registration. For any Holder that is a partnership, limited partnership, limited
liability company or corporation, the partners, limited partners, retired
partners, retired limited partners, members, retired members and stockholders of
such Holder, or the estates and family members of any such partners, limited
partners, retired partners, retired limited partners, members, retired members
and any trusts for the benefit of any of the foregoing persons shall be deemed
to be collectively a single "Holder," and any pro rata reduction with respect to
such "Holder" shall be based upon the aggregate amount of shares carrying
registration rights owned by all entities and individuals included in such
"Holder," as defined in this sentence.

            5.6   Furnishing Information. It shall be a condition precedent to
the obligations of the Company to take any action pursuant to Sections 5.1 and
5.2 above that the selling Holders shall furnish to the Company such information
regarding themselves, the Registrable Securities held by them and the intended
method of disposition of such securities as shall be required to effect the
registration of their Registrable Securities.

            5.7   Indemnification.

                  (a)   The Company will indemnify and hold harmless each
Holder, each of its officers, directors and partners, legal counsel, and
accountants and each person controlling such Holder within the meaning of
Section 15 of the Securities Act, with respect to which registration,
qualification or compliance has been effected pursuant to Section 5, and each
underwriter, if any, and each person who controls within the meaning of Section
15 of the Securities Act any underwriter, against all expenses, claims, losses,
damages, and liabilities (or actions, proceedings or settlements in respect
thereof) arising out of or based on (i) any untrue statement or alleged untrue
statement of a material fact contained in any prospectus, offering circular, or
other document, including any related registration statement, notification or
the like, incident to any such registration,

                                      11
<PAGE>

qualification or compliance, (ii) any omission or alleged omission to state
therein a material fact required to be stated therein or necessary to make the
statements therein not misleading, or (iii) any violation by the Company of the
Securities Act or any rule or regulation thereunder applicable to the Company
and relating to action or inaction required of the Company in connection with
any such registration, qualification or compliance, and will reimburse each such
Holder, each of its officers, directors, partners, legal counsel, and
accountants and each person controlling such Holder, each such underwriter and
each person who controls any such underwriter, for any legal and any other
expenses reasonably incurred in connection with investigating and defending or
settling any such claim, loss, damage, liability or action; provided, however,
that the Company will not be liable in any such case to the extent that any such
claim, loss, damage, liability or expense arises out of or is based on any
untrue statement or omission based upon written information furnished to the
Company by such Holder or underwriter. It is agreed that the indemnity agreement
contained in this Section 5.7(a) shall not apply to amounts paid in settlement
of any such loss, claim, damage, liability, or action if such settlement is
effected without the consent of the Company.

                  (b)   Each Holder will, if Registrable Securities held by such
Holder are included in the securities as to which such registration,
qualification or compliance is being effected, indemnify the Company, each of
its directors and officers, legal counsel and accountants, and each underwriter,
if any, of the Company's securities covered by such a registration statement,
each person who controls the Company or such underwriter within the meaning of
Section 15 of the Securities Act, each other such Holder and each of its
officers and directors, and each person controlling such other Holder, against
all claims, losses, damages and liabilities (or actions in respect thereof)
arising out of or based on (i) any untrue statement or alleged untrue statement
of a material fact by such Holder contained in any such registration statement,
prospectus, offering circular or other document, or (ii) any omission or alleged
omission to state therein by such Holder a material fact required to be stated
therein or necessary to make the statements therein not misleading, and will
reimburse the Company and such Holders, directors, officers, legal counsel,
accountants, persons, underwriters or control persons for any legal or any other
expenses reasonably incurred in connection with investigating or defending any
such claim, loss, damage, liability or action, in each case to the extent, but
only to the extent, that such untrue statement or alleged untrue statement or
omission or alleged omission is made in such registration statement, prospectus,
offering circular or other document in reliance upon and in conformity with
written information furnished to the Company by such Holder; provided, however,
that the obligations of such Holder hereunder shall not apply to amounts paid in
settlement of any such claims, losses, damages or liabilities (or actions in
respect thereof) if such settlement is effected without the consent of such
Holder; and provided that in no event shall any indemnity under this Section
5.7(b) exceed the net proceeds from the offering received by such Holder.

                  (c)   Each party entitled to indemnification under this
Section 5.7 (the "INDEMNIFIED PARTY") shall give notice to the party required to
provide indemnification (the "INDEMNIFYING PARTY") promptly after such
Indemnified Party has actual knowledge of any claim as to which indemnity may be
sought, and shall permit the Indemnifying Party to assume the defense of such
claim or any litigation resulting therefrom; provided, however, that legal
counsel for the Indemnifying Party, who shall conduct the defense of such claim
or any litigation resulting

                                      12
<PAGE>
therefrom, shall be approved by the Indemnified Party (whose approval shall not
be unreasonably withheld), and the Indemnified Party may participate in such
defense at such party's expense; and, provided further, however, that the
failure of any Indemnified Party to give notice as provided herein shall not
relieve the Indemnifying Party of its obligations under this Section 5.7(c), to
the extent such failure is not prejudicial. Consent by the Indemnifying Party to
entry of any judgment or entry into any settlement shall not bind the
Indemnified Party without the Indemnified Party's written consent, unless such
settlement includes as an unconditional term thereof the giving by the claimant
or plaintiff to the Indemnified Party of a release from all liability with
respect to such claim or litigation. Each Indemnified Party shall furnish such
information regarding itself or the claim in question as in Indemnifying Party
may reasonably request in writing and as shall be reasonably required in
connection with defense of such claim and litigation resulting therefrom.

                  (d)   If the indemnification provided for in this Section 5.7
is held by a court of competent jurisdiction to be unavailable to an Indemnified
Party with respect to any loss, liability, claim, damage or expense referred to
herein, then the Indemnifying Party, in lieu of indemnifying such Indemnified
Party hereunder, shall contribute to the amount paid or payable by such
Indemnified Party as a result of such loss, liability, claim, damage or expense
in such proportion as is appropriate to reflect the relative fault of the
Indemnifying Party on the one hand and of the Indemnified Party on the other in
connection with the statements or omissions that resulted in such loss,
liability, claim, damage or expense as well as any other relevant equitable
considerations. The relative fault of the Indemnifying Party and of the
Indemnified Party shall be determined by reference to, among other things,
whether the untrue or alleged untrue statement of a material fact or the
omission to state a material fact relates to information supplied by the
Indemnifying Party or by the Indemnified Party and to parties' relative intent,
knowledge, access to information, and opportunity to correct or prevent such
statement or omissions.

                  (e)   Notwithstanding the foregoing, to the extent that the
provisions on indemnification and contribution contained in the underwriting
agreement entered into in connection with an underwritten public offering of the
Company's securities are in conflict with the foregoing provisions, the
provisions in the underwriting agreement shall control.

            5.8   Transfer or Assignment of Registration Rights. The rights to
cause the Company to register Registrable Securities pursuant to this Section 5
may not be transferred or assigned by a Holder without the prior written consent
of the Company.

            5.9   Amendment of Registration Rights. Any provision of this
Section 5 may be amended and the observance thereof may be waived (either
generally or in a particular instance and either retroactively or
prospectively), only with the written consent of the Company and the holders of
at least a majority of the Registrable Securities then outstanding. Any
amendment or waiver effected in accordance with this Section 5.9 shall be
binding upon each Holder and the Company.

      6.    Management Rights.

            6.1   For so long as any Purchaser owns 1,000,000 shares of Voting
Stock, the Company covenants that:

                                      13
<PAGE>

                  (a)   if any Purchaser is not represented on the Board, such
Purchaser shall be entitled to consult with and advise management of the Company
on significant business issues, including management's proposed annual operating
plans, and management will meet with the Purchaser regularly during each year at
the Company's facilities at mutually agreeable times for such consultation and
advice and to review progress in achieving said plans;

                  (b)   each Purchaser may examine the books and records of the
Company and inspect its facilities and may request information at reasonable
times and intervals concerning the general status of the Company's financial
condition and operations, provided that access to highly confidential
proprietary information and facilities need not be provided; and

                  (c)   if any Purchaser is not represented on the Board, the
Company shall, concurrently with delivery to the Board, give a representative of
such Purchaser copies of all notices, minutes, consents and other material that
the Company provides to its directors, except that the representative may be
excluded from access to any material or meeting or portion thereof if the Board
determines in good faith, upon advice of counsel, that such exclusion is
reasonably necessary to preserve the attorney-client privilege, to protect
highly confidential proprietary information, or for other similar reasons,
provided that, upon reasonable notice, at a scheduled meeting of the Board or
such other time, if any, as the Board may determine in its sole discretion, such
representative may address the Board with respect to such Purchaser's concerns
regarding significant business issues facing the Company.

      7.    Miscellaneous.

            7.1   Governing Law; Jurisdiction. This Agreement shall be governed
by and construed and enforced in accordance with the internal laws of the State
of Delaware without giving effect to the principles of conflicts of laws. Any
legal action or other legal proceeding relating to this Agreement or the
enforcement of any provision of this Agreement may be brought or otherwise
commenced in any state or federal court located in the State of Delaware. Each
party hereto agrees to the entry of an order to enforce any resolution,
settlement, order or award made pursuant to this Section 7.1 by the state and
federal courts located in the State of Delaware and in connection therewith
hereby waives, and agrees not to assert by way of motion, as a defense, or
otherwise, any claim that such resolution, settlement, order or award is
inconsistent with or violative of the laws or public policy of the laws of the
State of Delaware or any other jurisdiction.

            7.2   Successors and Assigns. Except as otherwise expressly provided
herein, the provisions hereof shall inure to the benefit of, and be binding
upon, the successor and assigns of the parties hereto.

            7.3   Entire Agreement; Amendment. This Agreement and the Warrants
constitute the full and entire understanding and agreement between the parties
with regard to the subjects hereof and thereof. Neither this Agreement nor any
provision hereof may be amended, changed, waived, discharged or terminated other
than by a written instrument signed by the party against who enforcement of any
such amendment, change, waiver, discharge or termination is sought. This

                                      14
<PAGE>

Agreement shall become and thereafter be null and void if the Effective Date
shall not have occurred on or prior to April 30, 2004.

            7.4   Notices, etc. All notices and other communications required or
permitted hereunder shall be effective upon receipt and shall be in writing and
may be delivered in person, by telecopy, electronic mail, express delivery
service or U.S. mail, in which event it may be mailed by first-class, certified
or registered, postage prepaid, addressed, to the party to be notified, at the
respective addresses set forth below, or at such other address which may
hereinafter be designated in writing:

                  (a)   If to the Purchasers, to:

                        Warburg Pincus Private Equity VIII, L.P.
                        Warburg Pincus Netherlands Private Equity VIII I C.V.
                        Warburg Pincus Netherlands Private Equity VIII II C.V.
                        Warburg Pincus Germany Private Equity VIII K.G.
                        c/o Warburg Pincus LLC
                        466 Lexington Avenue
                        New York, NY 10017
                        Attention: Jeffrey A. Harris
                        Fax No. 212-878-6139

                        with a copy to:

                        Willkie Farr & Gallagher LLP
                        787 Seventh Avenue
                        New York, NY 10019
                        Attention: Michael A. Schwartz, Esq.
                        Fax No. 212-728-9267

                  (b)   If to the Company, to:

                        ScanSoft, Inc.
                        9 Centennial Drive
                        Peabody, MA 01960
                        Attention: Chief Executive Officer
                        Phone: (978) 977-2000
                        Fax:   (978) 977-2436

                        with a copy to:

                        Wilson Sonsini Goodrich & Rosati
                        650 Page Mill Road,
                        Palo Alto, CA 94304

                                      15
<PAGE>

                        Attention: Robert D. Sanchez, Esq.
                        Fax No. 650-493-6811

            7.5   Severability. If any provision of this Agreement shall be
judicially determined to be invalid, illegal or unenforceable, the validity,
legality and enforceability of the remaining provisions shall not in any way be
affected or impaired thereby.

            7.6   Titles and Subtitles. The titles of the Articles and Sections
of this Agreement are for convenience of reference only and in no way define,
limit, extend, or describe the scope of this Agreement or the intent of any of
its provisions.

            7.7   Counterparts. This Agreement may be executed in any number of
counterparts, each of which shall be an original, but all of which together
shall constitute one instrument.

            7.8   Delays or Omissions. It is agreed that no delay or omission to
exercise any right, power or remedy accruing to any party upon any breach or
default of any other party under this Agreement shall impair any such right,
power or remedy, nor shall it be construed to be a waiver of any such breach or
default, or any acquiescence therein, or of any similar breach or default
thereafter occurring; nor shall any waiver of any single breach or default be
deemed a waiver of any other breach or default theretofore or thereafter
occurring. It is further agreed that any waiver, permit, consent or approval of
any kind or character of any breach or default under this Agreement, or any
waiver of any provisions or conditions of this Agreement must be in writing and
shall be effective only to the extent specifically set forth in writing, and
that all remedies, either under this Agreement, by law or otherwise, shall be
cumulative and not alternative.

            7.9   Consents. Any permission, consent, or approval of any kind or
character under this Agreement shall be in writing and shall be effective only
to the extent specifically set forth in such writing.

            7.10  SPECIFIC PERFORMANCE. THE PARTIES HERETO AGREE THAT
IRREPARABLE DAMAGE WOULD OCCUR IN THE EVENT THAT ANY OF THE PROVISIONS OF THIS
AGREEMENT WERE NOT PERFORMED IN ACCORDANCE WITH ITS SPECIFIC INTENT OR WERE
OTHERWISE BREACHED. IT IS ACCORDINGLY AGREED THAT THE PARTIES SHALL BE ENTITLED
TO AN INJUNCTION OR INJUNCTIONS, WITHOUT BOND, TO PREVENT OR CURE BREACHES OF
THE PROVISIONS OF THIS AGREEMENT AND TO ENFORCE SPECIFICALLY THE TERMS AND
PROVISIONS HEREOF, THIS BEING IN ADDITION TO ANY OTHER REMEDY TO WHICH THEY MAY
BE ENTITLED BY LAW OR EQUITY, AND ANY PARTY SUED FOR BREACH OF THIS AGREEMENT
EXPRESSLY WAIVES ANY DEFENSE THAT A REMEDY IN DAMAGES WOULD BE ADEQUATE.

            7.11  Payment of Fees and Expenses. Each party shall be responsible
for paying its own fees, costs and expenses in connection with this Agreement
and the transactions herein contemplated.

                                      16
<PAGE>

            7.12  Construction of Agreement. No provision of this Agreement
shall be construed against either party as the drafter thereof.

            7.13  Section References. Unless otherwise stated, any reference
contained herein to a Section or subsection refers to the provisions of this
Agreement.

            7.14  Variations of Pronouns. All pronouns and all variations
thereof shall be deemed to refer to the masculine, feminine, or neuter, singular
or plural, as the context in which they are used may require.

                  [Remainder of Page Intentionally Left Blank]

                                      17
<PAGE>

IN WITNESS WHEREOF, the parties have caused this Stockholders Agreement to be
duly executed and delivered by their proper and duly authorized officers as of
the day and year first written above.

                                      SCANSOFT, INC.

                                      By: /s/ Paul Ricci
                                          -------------------------------
                                          Name:  Paul Ricci
                                          Title: Chief Executive Officer

                                      PURCHASERS

                                      WARBURG PINCUS PRIVATE EQUITY VIII, L.P.

                                      By: WARBURG PINCUS & CO.,
                                          its General Partner

                                      By: /s/ Jeffrey A. Harris
                                          -------------------------------
                                          Name:  Jeffrey A. Harris
                                          Title: Partner

                                      WARBURG PINCUS NETHERLANDS PRIVATE
                                         EQUITY VIII I C.V.

                                      By: WARBURG PINCUS & CO.,
                                          its General Partner

                                      By: /s/ Jeffrey A. Harris
                                          -------------------------------
                                          Name:  Jeffrey A. Harris
                                          Title: Partner

                                      18
<PAGE>

                                      WARBURG PINCUS NETHERLANDS PRIVATE
                                         EQUITY VIII II C.V.

                                      By: WARBURG PINCUS & CO.,
                                          its General Partner

                                      By: /s/ Jeffrey A. Harris
                                          -------------------------------
                                          Name:  Jeffrey A. Harris
                                          Title: Partner

                                      WARBURG PINCUS GERMANY PRIVATE
                                         EQUITY VIII, K.G.

                                      By: WARBURG PINCUS & CO.,
                                          its General Partner

                                      By: /s/ Jeffrey A. Harris
                                          -------------------------------
                                          Name:  Jeffrey A. Harris
                                          Title: Partner

                                      19