<PAGE>

                                                                    EXHIBIT 10.1

March 18, 2004

Stuart Patterson

Dear Stuart:

This letter is in response to your letter dated March 15, 2004 resigning as an
Executive Officer and Director of ScanSoft, Inc. (the "Company").

As a result of your resignation as an Executive Officer and Director of the
Company, the Compensation Committee of the Board of Directors has approved the
following separation package:

         1.       Your resignation as an Executive Officer and Director will be
                  effective March 22, 2004 and the effective date of your
                  termination will be March 22, 2004 (the "Separation Date").

         2.       Upon acceptance of this agreement, the Company will continue
                  to make salary payments to you, at your current rate of pay,
                  minus all applicable payroll deductions and withholding for a
                  period of eight (8) months and nine (9) days from the
                  Separation Date. Severance benefits will be paid in
                  semi-monthly installments in accordance with the Company's
                  current payroll schedule. You will receive your severance
                  benefit checks via direct deposit and a copy will be mailed to
                  your home address that we have on record.

         3.       The Company will pay for your health insurance continuation
                  benefits provided under COBRA as of the Separation Date, for a
                  period of eight (8) months and nine (9) days from the
                  Separation Date. Thereafter, you will be eligible to
                  participate in health insurance continuation benefits under
                  COBRA at your own expense.

         4.       The sixty four thousand five hundred (64,500) shares of
                  restricted stock granted to you on March 25, 2003 and the two
                  thousand nine hundred and ten (2,910) shares of restricted
                  stock granted to you on February 24, 2004 (collectively, the
                  "Restricted Stock") shall fully vest automatically without any
                  further action by either party on the one year anniversary of
                  the Separation Date, notwithstanding any terms of such grants
                  to the contrary, subject however to your compliance with
                  Section 9 below.

         5.       On the Separation Date you will be entitled to automatic
                  acceleration of sixty two thousand five hundred (62,500)
                  options to purchase shares of common stock granted to you on
                  August 11, 2003, notwithstanding the terms of any such grants
                  to the contrary.

<PAGE>

         6.       You will be able to exercise one hundred and eighty seven
                  thousand five hundred (187,500) options, which represents all
                  vested stock options (and which includes the sixty two
                  thousand five hundred (62,500) options accelerated pursuant to
                  Section 5), for a period of one (1) year following the
                  Separation Date, notwithstanding any terms of such grant to
                  the contrary.

         7.       The Company will reimburse you for all reasonable
                  out-of-pocket business expenses incurred by you on Company
                  business through the Separation Date in accordance with
                  company policy. In addition, on your Separation Date you will
                  be paid for all accrued and unused vacation days.

         8.       You agree that for a period of one (1) year after the
                  Separation Date, you will not, without the express written
                  consent of the Company, enter, engage in, participate in, or
                  assist, either as an individual on your own or as a partner,
                  joint venturer, employee, agent, consultant, officer, trustee,
                  director, owner, part-owner, shareholder, or in any other
                  capacity, in the United States of America, directly or
                  indirectly, any other business organization whose activities
                  or products are (a) related to speech systems, technologies,
                  and/or applications and (b) competitive with the activities or
                  products of the Company then existing or under development as
                  of the Separation Date. Nothing in this Agreement shall
                  prohibit you from working for such business organizations as
                  long as you do not work for or with the department, division,
                  or group in that employer's organization that is engaging in
                  such competitive activities or developing such competitive
                  products. Examples of such competitive business organizations
                  under this paragraph include the following business
                  organizations and their successors, affiliates, and assigns:
                  Nuance Communications, Inc., IBM Corporation's speech
                  division, Microsoft Corporation's speech division, and A.T.&T.
                  Corporation's speech division.

                  a.       You recognize that these restrictions on competition
                           are reasonable because of the Company's investment in
                           goodwill, its customer lists, and other proprietary
                           information and your knowledge of the Company's
                           business and business plans. If any period of time or
                           geographical area should be judged unreasonable in
                           any judicial proceeding, then the period of time or
                           geographical area shall be reduced to such extent as
                           may be deemed required so as to be reasonable and
                           enforceable.

                  b.       Nothing in this Agreement shall preclude you from
                           making passive investments of not more than two
                           percent (2%) of a class of securities of any business
                           enterprise registered under the Securities Exchange
                           Act of 1934, as amended.

         9.       You agree that you will not, either in your individual
                  capacity or on behalf of or through any other entity, either
                  directly or indirectly, hire, engage, recruit or participate
                  in any way in the hiring, engagement or recruitment of, or
                  participate in any effort to hire or solicit, any current or
                  future employees of ScanSoft or any subsidiary thereof (the
                  "ScanSoft Employees") for a period of one (1) year from the
                  Separation Date (the "No Hire Period"), without the prior
                  written consent of ScanSoft. For purposes of this Agreement,
                  ScanSoft Employees shall not include employees terminated by
                  ScanSoft.

                                      -2-
<PAGE>

                  You also agree not to be involved in any way in any
                  discussions with any third party with which you are affiliated
                  regarding the qualifications or employment of any ScanSoft
                  Employee, and if asked or approached, you shall express no
                  opinion regarding any ScanSoft Employee and shall state that
                  you are not at liberty to discuss such matters pursuant to the
                  terms of your Agreement with ScanSoft. If any ScanSoft
                  Employee is hired during the No Hire Period in what ScanSoft
                  believes is a violation of this Agreement, ScanSoft shall be
                  entitled to request, and you shall be obligated to provide, an
                  affidavit stating, under oath, that you had no involvement or
                  participation, either directly or indirectly, in the hiring,
                  engagement, recruitment, or solicitation of the ScanSoft
                  Employee at issue and that you did not participate in any act
                  to hire or solicit such employee.

                  If you violate this Section 9, you agree that you shall
                  forfeit all or a portion of the Restricted Stock on the terms
                  and conditions as set forth below:

                           (a) For a violation in connection with hiring,
                           solicitation or efforts to recruit any current or
                           future directors, vice presidents or senior vice
                           presidents of ScanSoft or any persons on a list of 45
                           names of key ScanSoft employees, such list to be
                           provided to you by ScanSoft by the Separation Date,
                           you shall forfeit your rights to all of the
                           Restricted Stock.

                           (b) For a violation in connection with the hiring,
                           solicitation or efforts to recruit any enterprise,
                           OEM and channel speech salesperson, you shall forfeit
                           your rights to 50% of the Restricted Stock.

                           (c) For a violation in connection with the hiring,
                           solicitation or efforts to recruit any other ScanSoft
                           Employee (the "Remaining Employees"), ScanSoft shall
                           be entitled to seek damages resulting from such
                           violation of this provision but there shall be no
                           forfeiture of the Restricted Stock as a result of the
                           first such violation of this subparagraph (c). If you
                           violate this subparagraph in connection with the
                           hiring, solicitation or efforts to recruit a second
                           of the Remaining Employees, you shall forfeit 50% of
                           the Restricted Stock. If you violate this
                           subparagraph (c) in connection with the hiring,
                           solicitation or efforts to recruit a third or any
                           other Remaining Employee, you shall forfeit 100% of
                           the Restricted Stock.

         10.      Both during and after your employment you acknowledge your
                  continuing obligations not to use or disclose any confidential
                  or proprietary information of the Company without prior
                  written authorization from a duly authorized representative of
                  the Company. You agree that all information, whether or not in
                  writing, of a private, secret or confidential nature
                  concerning the Company's business, business relationships,
                  products, customers, technology, research and development or
                  financial affairs (the "Confidential and Proprietary
                  Information") is and shall be the exclusive property of the
                  Company. By way of illustration, but not limitation,
                  Confidential and Proprietary Information may include
                  inventions, products, processes, applications, methods,
                  techniques, formulas, composition, compounds, projects,
                  developments, plans, research, data, financial data,

                                      -3-
<PAGE>

                  personnel data, computer programs, customer and supplier lists
                  and contacts at or knowledge of customers or prospective
                  customers of the Company.

                  You agree that all files, letters, memoranda, reports,
                  records, data, sketches, drawings, or other written,
                  photographic or other tangible material containing
                  confidential and Proprietary Information, whether created by
                  you or others, which shall come into your custody or
                  possession shall be and are the exclusive property of the
                  Company and shall be returned to the Company on the
                  termination of your employment. After such delivery, you agree
                  that you will not retain any such materials or copies thereof
                  or any such tangible property.

         11.      This Agreement shall be governed by Massachusetts law. Any
                  action brought to enforce this Agreement shall be brought in a
                  Massachusetts state or federal court, and each party to this
                  Agreement hereby consents to personal jurisdiction over him or
                  it in a Massachusetts state or federal court.

In exchange for the salary continuation and other consideration under this
Agreement, and except as otherwise set forth in this Agreement, you hereby
release, acquit and forever discharge the Company, its parents and subsidiaries,
and its and their respective officers, directors, agents, servants, employees,
attorneys, shareholders, successors, assigns and affiliates, of and from any and
all claims, liabilities, demands, causes of action, costs, expenses, attorneys'
fees, damages, indemnities and obligations of every kind and nature, in law,
equity, or otherwise, known and unknown, suspected and unsuspected, disclosed
and undisclosed, arising out of or in any way related to agreements, events,
acts or conduct at any time prior to and including the date you sign this
Agreement, including but not limited to: all such claims and demands directly or
indirectly arising out of or in any way connected with your employment with the
Company or the termination of that employment; claims or demands related to
salary, bonuses, commissions, stock, stock options, or any other ownership
interests in the Company, vacation pay, fringe benefits, expense reimbursements,
severance pay, or any other form of compensation; claims pursuant to any
federal, state or local law, statute or cause of action including, but not
limited to, the federal Civil Rights Act of 1964, as amended; the Federal Age
Discrimination in Employment Act of 1967, as amended (the "ADEA"); the federal
Americans with Disabilities Act of 1990; the Massachusetts General Laws, c151B;
the Massachusetts Fair Employment Practices Act, and any other applicable law,
rule, regulation or guideline; tort law; contract law; wrongful discharge;
discrimination; harassment; fraud; defamation; emotional distress; and breach of
the implied covenant of good faith and fair dealing.

You acknowledge that you are knowingly and voluntarily waiving and releasing any
rights you may have under the ADEA, as amended. You also acknowledge that the
consideration given for the waiver and release in the preceding paragraph hereof
is in addition to anything of value to which you were already entitled. You
further acknowledge that you have been advised by this writing, as required by
the ADEA, that: (a) your waiver and release do not apply to any rights or claims
that may arise after the execution date of this Agreement; (b) you have been
advised hereby that you have the right to consult with an attorney prior to
executing this Agreement; (c) you have forty-five (45) days to consider this
Agreement (although you may choose to voluntarily execute this Agreement
earlier); (d) you have seven (7) days following the execution

                                       -4-
<PAGE>

of this Agreement by the parties to revoke the Agreement; and (e) this Agreement
shall not be effective until the date upon which the revocation period has
expired, which shall be the eighth day after this Agreement is executed by you,
provided that the Company has also executed this Agreement by that date.

Neither you nor the Company will make any statements that are professionally or
personally disparaging about, or adverse to, the interests of the other,
including but not limited to, any statements that disparage any person, product,
service, finances, financial condition, capability or any other aspect of the
other party. Neither you nor the Company will engage in any conduct which is
intended to harm professionally or personally the reputation of the other party.

ScanSoft will reimburse you for your actual legal fees incurred in connection
with the preparation and negotiation of this Agreement, as demonstrated by
detailed invoices from your counsel, up to a maximum of six thousand dollars
($6,000).

Based on mutual agreement, after you cease to be an employee of the Company, the
Company may wish to engage your services for specific project based work. If
such services are engaged, a mutually agreed upon compensation will be paid, but
it will not affect any terms of this agreement.

Upon acceptance of this agreement, the Compensation Committee of the Board of
Directors has authorized this agreement to supercede all previous employment
agreements (however the terms of your Stock Option Agreement and Restricted
Stock and the corresponding plans shall remain in full force and effect, except
as specifically modified herein). Please sign below to indicate your acceptance.

Sincerely,                                           Agreed to:

/s/ Paul Ricci                                       /s/ Stuart Patterson
---------------                                      -------------------------
Paul Ricci, CEO                                      Stuart Patterson, President

                                       -5-