<PAGE>

                                                                   Exhibit 10.31

May 10, 2001

Mark McIlvane
c/o Clarent Corporation
700 Chesapeake Drive
Redwood City, CA  94063

Dear Mark:

I am pleased that we have been able to work out a mutually beneficial
transition.  This letter sets forth the substance of the separation agreement
(the "Agreement") to which you and Clarent Corporation (the "Company") have
agreed.

     1.  Separation Date.  You will resign from your position as the Company's
Executive Vice President, effective as of the Effective Date of this Agreement,
as set forth in paragraph 13 below, and you will resign from your employment
with the Company effective December 31, 2001 (the "Separation Date").

     2.  Transition Period. You will continue to report to work full time
through July 31, 2001 (the "Transition Date"), except for any vacation days you
take prior to that date. During the period after the Transition Date and until
the Separation Date (the "Transition Period"), your responsibilities shall be in
any area of your expertise, at the direction of Barry Forman (or his designee or
successor), at the Company's sole discretion. During the Transition Period, the
Company will continue to pay you your base salary as of the Transition Date
("Base Salary"), less standard payroll deductions and withholdings. You will not
receive any bonuses or commissions, and will not accrue any vacation time,
during the Transition Period.

     3.  Accrued Salary and Vacation. On the Separation Date, the Company will
pay you all accrued and unpaid salary earned through the Separation Date, and
all accrued and unused vacation as of the Transition Date, less standard payroll
deductions and withholdings.

     4.  Severance Payment. In exchange for the Supplemental Release described
in paragraph 14 below, the Company will make a severance payment to you in an
amount equal to seven (7) months of your Base Salary, less standard payroll
deductions and withholdings. The Company will make this payment on or about
January 15, 2002, or the effective date of the Supplemental Release, whichever
is later.

     5.  Stock Options. Your Company stock options (the "Options") will continue
to vest until and ending on the Separation Date. The Options will continue to be
governed by the applicable stock option grants, stock option agreements and
stock option plan. In particular, you have the right to exercise all vested
shares of the Options for three (3) months after the Separation Date.
<PAGE>

Mark McIlvane
May 10, 2001
Page 2

     6.  Health Insurance; Other Benefits. You will remain eligible for coverage
under the Company's current health insurance policies through and until the
Transition Date. You will be eligible to continue your group health insurance
benefits after the Transition Date, to the extent provided by federal COBRA law
and the Company's current group health insurance policies. If you elect
continued coverage under COBRA, the Company, as part of this Agreement, will pay
your COBRA premiums through the coverage period ending on the Separation Date.
Any additional insurance coverages under which you are eligible to receive
benefits that are in effect as of the Effective Date shall continue through the
expiration of their current policy periods, and will not be renewed by the
Company on your behalf.

     7.  Other Compensation or Benefits. You acknowledge that you will not
receive from the Company any additional compensation (including but not limited
to salary, bonuses, commissions or stock option grants), severance or benefits
after the Separation Date, except as expressly provided in this Agreement.

     8.  Expense Reimbursements. You agree that within thirty (30) business days
after the Transition Date you will submit your final documented expense
reimbursement statement reflecting all business expenses you have incurred
through the Separation Date, if any, for which you seek reimbursement. The
Company will reimburse you for these expenses pursuant to its regular business
practices. You will promptly submit for reimbursement any further statements
reflecting business expenses you incur during the Transition Period.

     9.  Return of Company Property. You agree to return to the Company, not
later than ten (10) business days after the Transition Date, all Company
documents (and all copies thereof) and other Company property that you have had
in your possession at any time (collectively, "Company Material"), including,
but not limited to, Company files, notes, drawings, records, business plans and
forecasts, financial information, specifications, training materials, computer-
recorded information, tangible property including, but not limited to,
computers, credit cards, entry cards, identification badges and keys; and any
materials of any kind that contain or embody any proprietary or confidential
information of the Company (and all reproductions thereof); provided, however,
that you may retain your Company laptop computer, so long as you permit the
Company, not later than ten (10) days after the Transition Date, to remove all
Company Material from the laptop, except any such material the Company
determines is needed for the performance of your duties to the Company. During
the Transition Period, you may receive new Company Material as necessary to
perform your responsibilities during the Transition Period, and you agree to
return promptly all such Company Material when no longer needed to perform such
responsibilities.

     10. Proprietary Information. You agree to sign, not later than the
Effective Date, and to fully comply with, the Company's standard Employee
Proprietary Information and Inventions Agreement, in the form attached as
Exhibit A hereto. Among other obligations in such agreement, you agree not to
use or disclose, at any time, any confidential or proprietary
<PAGE>

Mark McIlvane
May 10, 2001
Page 3

information of the Company without prior written authorization from a duly
authorized representative of the Company, except in the performance of your
duties to the Company.

     11.  Nondisparagement. Both you and the Company (by its officers and
directors) agree not to disparage the other party, or the other party's
officers, directors, employees, shareholders, affiliates, and agents, in any
manner likely to be harmful to them or their business, business reputation or
personal reputation; provided, however, that both you and the Company shall
respond accurately and fully to any question, inquiry or request for information
when required by legal process.

     12.  Release of Claims.  In exchange for the consideration provided in
paragraphs 1 and 2 above, to which you would not otherwise be entitled, you
hereby agree to release, acquit and forever discharge the Company and its
officers, directors, agents, employees, attorneys, shareholders, successors,
assigns and affiliates, of and from any and all claims, liabilities, demands,
causes of action, costs, expenses, attorneys fees, damages, indemnities and
obligations of every kind and nature, in law, equity, or otherwise, known and
unknown, suspected and unsuspected, disclosed and undisclosed, arising out of or
in any way related to agreements, events, acts or conduct at any time prior to
and including the date you sign this Agreement, including but not limited to:
any and all such claims and demands directly or indirectly arising out of or in
any way connected with your employment with the Company or the termination of
that employment; claims or demands related to salary, bonuses, commissions,
stock, stock options, or any other ownership interests in the Company, vacation
pay, fringe benefits, expense reimbursements, sabbatical benefits, severance
benefits, or any other form of compensation; claims pursuant to any federal,
state, local law, statute or cause of action including, but not limited to, the
federal Civil Rights Act of 1964, as amended; the federal Americans with
Disabilities Act of 1990; the federal Age Discrimination in Employment Act of
1967, as amended ("ADEA"); the California Fair Employment and Housing Act, as
amended; tort law; contract law; wrongful discharge; discrimination; fraud;
defamation; harassment; emotional distress; and breach of the implied covenant
of good faith and fair dealing.  The releases given herein shall not bar any
claim for breach of the terms of this Agreement.

     13.  ADEA Waiver.  You acknowledge that you are knowingly and voluntarily
waiving and releasing any rights you may have under the ADEA.  You also
acknowledge that the consideration given for the waiver and release in the
preceding paragraph is in addition to anything of value to which you were
already entitled.  You further acknowledge that you have been advised by this
writing, as required by the ADEA, that:  (a) your waiver and release do not
apply to any rights or claims that arise after the date you sign this Agreement;
(b) you should consult with an attorney prior to executing this Agreement; (c)
you have twenty-one (21) days to consider this Agreement (although you may
choose to voluntarily execute this Agreement earlier); (d) you have seven (7)
days following the date you sign this Agreement to revoke the Agreement; and (e)
this Agreement will not be effective until the date upon which the revocation
<PAGE>

Mark McIlvane
May 10, 2001
Page 4

period has expired, which will be the eighth day after you sign this Agreement
(the "Effective Date").

     14.  Supplemental Release. In consideration of and as a condition to
receiving the payment provided in paragraph 4 above, to which you would not
otherwise be entitled, you agree to sign and return to the Company a
supplemental release of claims (the "Supplemental Release"), in the form
attached hereto as Exhibit B, on or after the Separation Date.

     15.  Waiver.  In granting the release herein, you acknowledge that you
understand that you are waiving the benefit of California Civil Code section
1542, which states:

     A general release does not extend to claims which the creditor does not
     know or suspect to exist in his favor at the time of executing the release,
     which if known by him must have materially affected his settlement with the
     debtor.

You hereby expressly waive and relinquish all rights and benefits under that
section and any law or legal principle of similar effect in any jurisdiction
with respect to the release of unknown and unsuspected claims granted in this
Agreement.

     16.  Confidentiality.  The provisions of this Agreement will be held in
strictest confidence by you and the Company and will not be publicized or
disclosed in any manner whatsoever; provided, however, that:  (a) you may
disclose this Agreement to your immediate family; (b) the parties may disclose
this Agreement in confidence to their respective attorneys, accountants,
auditors, tax preparers, and financial advisors; (c) the Company may disclose
this Agreement as necessary to fulfill standard or legally required corporate
reporting or disclosure requirements; and (d) the parties may disclose this
Agreement insofar as such disclosure may be necessary to enforce its terms or as
otherwise required by law.  In particular, and without limitation, you will not
disclose the provisions of this Agreement to any current or former Company
employee or any other Company personnel, except to the extent necessary to
enforce its terms.

     17.  Dispute Resolution. To ensure rapid and economical resolution of any
and all disputes that may arise in connection with this Agreement, you and the
Company agree that any and all disputes, claims, and causes of action, in law or
equity, arising from or relating to this Agreement or its enforcement,
performance, breach, or interpretation, will be resolved solely and exclusively
by final, binding, and confidential arbitration, by a single arbitrator, in San
Francisco, California, and conducted by Judicial Arbitration & Mediation
Services, Inc. ("JAMS") under its then-existing employment rules and procedures.
Nothing in this section, however, is intended to prevent either party from
obtaining injunctive relief in court to prevent irreparable harm pending the
conclusion of any such arbitration.

     18.  Entire Agreement. This Agreement, including all exhibits, constitutes
the complete, final and exclusive embodiment of the entire agreement between you
and the

<PAGE>

Mark McIlvane
May 10, 2001
Page 5

Company with regard to the subject matters hereof. Except as provided herein,
the Agreement supersedes and merges any and all agreements entered into by and
between you and the Company, including but not limited to the letter agreements
between you and the Company dated June 9, 1997 and June 1, 1998, attached hereto
as Exhibits C and D, respectively. This Agreement is entered into without
reliance on any promise or representation, written or oral, other than those
expressly contained herein. It may not be modified except in a writing signed by
you and a duly authorized officer of the Company. Each party has carefully read
this Agreement, has been afforded the opportunity to be advised of its meaning
and consequences by his or its respective attorneys, and signed the same of his
or its own free will.

     19.  Successors and Assigns.  This Agreement will bind the heirs, personal
representatives, successors, assigns, executors and administrators of each
party, and will inure to the benefit of each party, its heirs, successors and
assigns.

     20.  Applicable Law. This Agreement will be deemed to have been entered
into and will be construed and enforced in accordance with the laws of the State
of California as applied to contracts made and to be performed entirely within
California, without regard to conflicts of laws.

     21.  Severability. If a court of competent jurisdiction determines that any
term or provision of this Agreement is invalid or unenforceable, in whole or in
part, then the remaining terms and provisions hereof will be unimpaired. The
court or arbitrator will then have the authority to modify or replace the
invalid or unenforceable term or provision with a valid and enforceable term or
provision that most accurately represents the parties' intention with respect to
the invalid or unenforceable term or provision.

                 [REMAINDER OF PAGE INTENTIONALLY LEFT BLANK]
<PAGE>

Mark McIlvane
May 10, 2001
Page 6

     22.   Counterparts.  This Agreement may be executed in two counterparts,
each of which will be deemed an original, all of which together constitutes one
and the same instrument.

If this Agreement is acceptable to you, please sign below and return the
original to me.  Please sign and return the Supplemental Release (Exhibit B) to
me on or after the Separation Date.

I wish you luck in your future endeavors.

Sincerely,

Clarent Corporation

By: /s/ Joanne Webster
    ------------------------------------------
    Joanne Webster
    Vice President, Worldwide Human Resources

Exhibit A  Employee Proprietary Information and Inventions Agreement
Exhibit B  Supplemental Release
Exhibit C  June 9, 1997 Letter Agreement
Exhibit D  June 1, 1998 Letter Agreement

Understood and Agreed:

/s/ Mark McIlvane
-----------------
Mark McIlvane

Date:  05/14/2001
       ----------
<PAGE>

                                   Exhibit A

           EMPLOYEE PROPRIETARY INFORMATION AND INVENTIONS AGREEMENT
<PAGE>

                                   Exhibit B

                             SUPPLEMENTAL RELEASE

                 (To be signed on or after December 31, 2001)

In further consideration of the separation agreement between Clarent Corporation
(the "Company") and me dated May 10, 2001, and in particular the severance
payment provided in paragraph 4 thereof, I hereby release, acquit and forever
discharge the Company, its officers, directors, agents, servants, employees,
attorneys, shareholders, successors, assigns and affiliates, of and from any and
all claims, liabilities, demands, causes of action, costs, expenses, attorneys
fees, damages, indemnities and obligations of every kind and nature, in law,
equity, or otherwise, known and unknown, suspected and unsuspected, disclosed
and undisclosed, arising out of or in any way related to agreements, events,
acts or conduct at any time prior to and including the date I sign this
Supplemental Release ("Release"), including but not limited to:  all such claims
and demands directly or indirectly arising out of or in any way connected with
my employment with the Company or the termination of that employment; claims or
demands related to salary, bonuses, commissions, stock, stock options, or any
other ownership interests in the Company, vacation pay, fringe benefits, expense
reimbursements, severance pay, or any other form of compensation; claims
pursuant to any federal, state or local law, statute, or cause of action
including, but not limited to, the federal Civil Rights Act of 1964, as amended;
the federal Americans with Disabilities Act of 1990; the federal Age
Discrimination in Employment Act of 1967, as amended ("ADEA"); the California
Fair Employment and Housing Act, as amended; tort law; contract law; wrongful
discharge; discrimination; harassment; fraud; defamation; emotional distress;
and breach of the implied covenant of good faith and fair dealing.

I acknowledge that I am knowingly and voluntarily waiving and releasing any
rights I may have under the ADEA, as amended.  I also acknowledge that the
consideration given for the waiver and release in the preceding paragraph hereof
is in addition to anything of value to which I was already entitled.  I further
acknowledge that I have been advised by this writing, as required by the ADEA,
that:  (a) my waiver and release do not apply to any rights or claims that may
arise after the date I sign this Release; (b) I have been advised hereby that I
have the right to consult with an attorney prior to executing this Release; (c)
I have had twenty-one (21) days to consider this Release; (d) I have seven (7)
days following the date I sign this Release to revoke the Release; and (e) this
Release will not be effective until the date upon which the revocation period
has expired, which will be the eighth day after this Release is executed by me.

I UNDERSTAND THAT THIS AGREEMENT INCLUDES A RELEASE OF ALL KNOWN AND UNKNOWN
CLAIMS.  In giving this release, which includes claims which may be unknown to
me at present, I acknowledge that I have read and understand Section 1542 of the
California Civil Code, which states:  "A general release does not extend to
claims which the creditor does not know or suspect to exist in his favor at the
time of executing the release, which if known by him must have materially
affected his settlement with the debtor." I hereby expressly waive and
relinquish all rights and benefits under that section and any law of

                                      1.
<PAGE>

any jurisdiction of similar effect with respect to my release of any unknown or
unsuspected claims I may have against the Company.

                                           By:______________________________
                                                Mark McIlvane

                                           Date:____________________________

                                      2.
<PAGE>

                                   Exhibit C

                         JUNE 9, 1997 LETTER AGREEMENT
<PAGE>

                                   Exhibit D

                         JUNE 1, 1998 LETTER AGREEMENT