<PAGE>
                                                                    Exhibit 4.23

                         CONTROL DELIVERY SYSTEMS, INC.
                               SEVERANCE AGREEMENT

     AGREEMENT made this 20th day of February, 2004, by and between Paul Ashton
("Executive") and Control Delivery Systems, Inc. (the "Company").

     Whereas, the Board of Directors of the Company (the "Board") recognizes
that the possibility of additional layoffs and/or hibernation exists and that
such possibility, and the uncertainty and questions which it may raise among
certain key management personnel, may result in the departure or distraction of
such management personnel to the detriment of the Company and its stockholders;

     Whereas, the Board has determined that appropriate steps should be taken to
reinforce and encourage the continued attention and dedication of certain key
members of the Company's management, including Executive, to their duties, to
assist the Board and the Company in maximizing the value of the Company, without
distraction; and

     Whereas, the Board wishes to induce Executive and other key members of
management to remain in the employ of the Company and to assure them of fair
severance should the employment of any of them terminate on or before October 7,
2005;

NOW, THEREFORE, in consideration of the premises and the mutual promises, terms
and conditions contained herein, the parties hereto agree as follows:

1.   Severance Benefits. If on or before October 7, 2005 the Company terminates
     Executive's employment without Cause, or if Executive terminates his or her
     employment for Good Reason, the Company will provide severance benefits to
     Executive as follows:

     a.   All of the shares of restricted stock granted to Executive will
          automatically and immediately vest upon such termination.

     b.   All options to purchase Company stock held by Executive will
          automatically and immediately vest and become exercisable upon such
          termination and remain exercisable for a period of six months
          following such termination (or three months in the case of incentive
          stock options).

     c.   The Company will pay to Executive within 30 days of the termination a
          lump-sum cash amount equal to the sum of (x) an amount equal to six
          months' Base Salary plus (y) a pro-rated portion of the maximum bonus
          that would otherwise be payable in the year of termination, if any.
          "Base Salary" means Executive's base salary on the date of this
          agreement. For purposes of this Agreement, the maximum bonus payable
          in any year will be calculated assuming all bonus targets or formulas
          for determining the bonus in such year had been met if Executive and
          Board had, prior to the termination of Executive's employment, agreed
          on such targets or formulas. If no such targets or formulas have been
          set as of such termination date, then the

<PAGE>

          maximum bonus shall be deemed to be the actual bonus paid to Executive
          during the preceding fiscal year.

     d.   The Company will continue for a period of six months from the date of
          termination to provide Executive with the following benefits: (1) the
          Company's group medical plan, and (2) life insurance arrangements or
          disability plan arrangements provided to executive-level employees of
          the Company. To the extent that the Company is unable to provide such
          benefits to Executive under its existing plans and arrangements, it
          will pay Executive cash amounts equal to the cost the Company would
          have incurred to provide those benefits.

2.   Definitions.

     a.   "Cause" means only (a) willful malfeasance or gross negligence in the
          performance by Executive of his or her duties, resulting in material
          harm to the Company, (b) fraud or dishonesty by Executive with respect
          to the Company, or (c) Executive's conviction of any felony involving
          deception, fraud or moral turpitude. The Company may treat a
          termination of Executive's employment as termination for Cause only
          after (i) giving Executive written notice of the intention to
          terminate for Cause and of his or her right to a hearing by the Board,
          (ii) at least 30 days after giving the notice, conducting a hearing by
          the Board at which Executive may be represented by counsel, and (iii)
          giving Executive 30 days' written notice of the results of the
          hearing, which shall require a vote of a majority of the Directors
          then in office other than Executive. For purposes of this definition
          of Cause, no act or omission shall be considered to have been
          "willful" unless it was not in good faith and Executive had knowledge
          at the time that the act or omission was not in the best interest of
          the Company. Any act or failure to act based on authority given
          pursuant to a resolution duly adopted by the Board or based on the
          advice of counsel of the Company shall be conclusively presumed to be
          done, or omitted to be done, by Executive in good faith and in the
          best interest of the Company. Cause shall not include willful failure
          due to incapacity resulting from physical or mental illness or any
          actual or anticipated failure after Notice of Termination for Good
          Reason.

     b.   "Good Reason" means (i) failure by the Company to maintain Executive
          in the positions of Chief Executive Officer and President including
          holding the title of and serving as Chief Executive Officer and
          President of the Company and having responsibility for performing all
          duties typically undertaken by such offices or assignment to Executive
          of duties materially inconsistent with such positions, (ii) the
          diminution in any material respect of Executive's positions or
          authority, excluding for this purpose an isolated, insubstantial and
          inadvertent action not taken in bad faith and which is remedied by the
          Company promptly after receipt of notice thereof given by Executive,
          (iii) failure by the Company to provide Executive with Base Salary,
          benefits and other compensation (including without limitation bonus
          and other incentive compensation) Executive was receiving as of the
          date of this Agreement, or (iv) relocation of Executive's principal
          place of

<PAGE>

          work to a location more than 30 miles from its current location
          without Executive's consent. For purposes of this definition of Good
          Reason, "Company" means Control Delivery Systems and any successor of
          Control Delivery Systems, provided if any such successor has a parent,
          then Company shall mean the ultimate parent corporation.

3.   Change of Control. Notwithstanding anything in this Agreement to the
     contrary (including without limitation Sections 1, 2 and 11 of this
     Agreement), if Executive is eligible for the benefits prescribed in the
     Change of Control Agreement between Executive and Company of even date
     herewith (the "Change of Control Agreement"), then Executive shall receive
     the benefits prescribed in the Change of Control Agreement and will not be
     eligible for benefits under this Agreement and the Company will have no
     obligation or liability to Executive under this Agreement.

4.   Terminability of Employment. This Agreement does not constitute a contract
     of employment for a specific term. Employment with the Company is at-will.
     The Company may at any time terminate Executive's employment with the
     Company with or without notice or Cause.

5.   Termination by the Company for Cause or by Executive Without Good Reason.
     If the Company terminates Executive's employment for Cause or if Executive
     terminates his employment other than for Good Reason, the Company will have
     no obligation or liability to Executive under this Agreement.

6.   Withholding. All payments required to be made by the Company to Executive
     under this Agreement will be subject to the withholding of such amounts, if
     any, relating to tax and other payroll deductions as may be required by
     law.

7.   Confidentiality, Non-Competition and Exclusivity. Executive further agrees
     to abide by all of the terms of the Confidentiality and Inventions
     Agreement dated as of July 27, 2001 ("Confidentiality and Inventions
     Agreement"). The parties recognize and agree that should the Company be
     required to pursue a claim against Executive under the Confidentiality and
     Inventions Agreement, the Company will likely be required to seek
     injunctive relief as well as damages at law. Accordingly, Section 8,
     Arbitration, will not apply to any action by the Company against Executive
     under the Confidentiality and Inventions Agreement.

8.   Arbitration. Except as otherwise provided in Section 7, any dispute or
     controversy between the parties involving the construction or application
     of this Agreement, or any claim arising out of or relating to this
     Agreement, or any claim arising out of or relating to Executive's
     employment by the Company that is not resolved within ten days by the
     parties will be settled by arbitration in Boston, Massachusetts, in
     accordance with the rules of the American Arbitration Association then in
     effect, and judgment upon the award rendered by the arbitrator(s) may be
     entered in any court having jurisdiction thereof. The Company and Executive
     agree that the arbitrator(s) will have no authority to award punitive or
     exemplary damages or so-called

<PAGE>

     consequential or remote damages such as damages for emotional distress. Any
     decision of the arbitrator(s) will be final and binding upon the parties.
     Either party may request that the arbitrator(s) submit written findings of
     fact and conclusions of law. The parties agree and understand that they
     hereby waive their rights to a jury trial of any dispute or controversy
     relating to the matters specified above in this Section.

9.   Termination upon Death or Disability. If Executive ceases to be an employee
     of the Company as a result of death or disability, the Company will have no
     further obligation or liability to Executive under this Agreement. However,
     nothing in this Agreement is intended to interfere with the rights of
     Executive and his family or beneficiaries under other applicable plans,
     policies or arrangements of the Company applicable in the event of death or
     disability.

10.  Rights of Survivors If Executive dies after becoming entitled to benefits
     under Section 1 following termination of employment but before all such
     benefits have been provided, all unpaid cash amounts will be paid to the
     beneficiary that has been designated by Executive in writing (the
     "beneficiary"), or if none, to Executive's estate.

11.  Successors. This Agreement will inure to and be binding upon the Company's
     successors. The Company will require any successor to all or substantially
     all of the business and/or assets of the Company by sale, merger or
     consolidation (where the Company is not the surviving corporation), lease
     or otherwise, to assume this Agreement. This Agreement is not otherwise
     assignable by the Company or the Executive.

12.  Amendment or Modification; Waiver. This Agreement may not be amended unless
     agreed to in writing by Executive and an expressly authorized officer of
     the Company. No waiver by either party of any breach of this Agreement will
     be deemed a waiver of a subsequent breach.

13.  Controlling Law. This Agreement will be controlled and interpreted pursuant
     to Massachusetts law, other than its choice of law principles.

14.  Entire Agreement. This Agreement constitutes the entire agreement and
     supersedes conflicting terms in all prior agreements and undertakings, both
     written and oral, between Executive and the Company with respect to the
     subject matter hereof. Except as otherwise set forth herein, the terms and
     conditions of all other agreements with Executive, including without
     limitation all Restricted Stock Award Agreements, Stock Option Grant
     Agreements, the Change of Control Agreement and the Confidentiality and
     Inventions Agreement, shall continue in full force and effect.

15.  Notices. Any notices required or permitted to be sent under this Agreement
     shall be effective when delivered by hand or mailed by registered or
     certified mail, return receipt requested, and addressed as follows:

<PAGE>

     If to the Company:

          Control Delivery Systems, Inc.
          400 Pleasant Street
          Watertown, MA 02472

     If to Executive:

          Paul Ashton
          19 Brimmer Street
          Boston, MA 02108

     Either party may change its address for receiving notices by giving notice
     to the other party.

16.  Release. Notwithstanding anything to the contrary contained in this
     Agreement, in order for Executive to be eligible for the severance benefits
     to be provided in accordance with this Agreement, Executive must sign (and
     not revoke within seven days of signing) a release of claims in the form
     attached hereto and marked Exhibit A.

            [THE REMAINDER OF THIS PAGE IS INTENTIONALLY LEFT BLANK.]

<PAGE>

     In witness whereof, the parties hereto have executed this Agreement as of
the date first set forth above.


                                        By: /s/ Paul Ashton
                                            ------------------------------------
                                        Name: Paul Ashton


                                        CONTROL DELIVERY SYSTEMS, INC.


                                        By: /s/ Lori H. Freedman
                                            ------------------------------------
                                        Name: Lori H. Freedman
                                        Title: Vice President, Corporate Affairs
                                               & General Counsel

<PAGE>

                                    EXHIBIT A

                                RELEASE OF CLAIMS

     FOR AND IN CONSIDERATION OF the benefits to be provided me in connection
with the termination of my employment, as set forth in the Severance Agreement
between me and Control Delivery Systems, Inc. (the "Company") dated as of
February 20, 2004 (the "Agreement"), which benefits are subject to my signing of
this Release of Claims and to which I am not otherwise entitled, and for other
good and valuable consideration, the receipt and sufficiency of which is hereby
acknowledged, I, on my own behalf and on behalf of my heirs, executives,
administrators, beneficiaries, representatives and assigns, and all others
connected with me, hereby release and forever discharge the Company, its
subsidiaries and other affiliates and all of their respective past, present and
future officers, directors, trustees, shareholders, employees, agents, general
and limited partners, members, managers, joint venturers, representatives,
successors and assigns, and all others connected with any of them, both
individually and in their official capacities, from any and all causes of
action, rights and claims of any type or description, known or unknown, which I
have had in the past, now have, or might now have, through the date of my
signing of this Release of Claims, in any way resulting from, arising out of or
connected with my employment by the Company or any of its subsidiaries or other
affiliates or the termination of that employment or pursuant to any federal,
state or local law, regulation or other requirement (including without
limitation Title VII of the Civil Rights Act of 1964, the Age Discrimination in
Employment Act, the Americans with Disabilities Act, and the fair employment
practices laws of the state or states in which I have been employed by the
Company or any of the subsidiaries or other affiliates, each as amended from
time to time).

     Excluded from the scope of this Release of Claims is (i) any claim arising
under the terms of the Agreement and (ii) any right of indemnification or
contribution that I have pursuant to the Articles of Incorporation or By-Laws of
the Company or any of its subsidiaries or other affiliates.

     In signing this Release of Claims, I acknowledge my understanding that I
may not sign it prior to the termination of my employment, but that I may
consider the terms of this Release of Claims for up to twenty-one (21) days (or
such longer period as the Company may specify) from the later of the date my
employment with the Company terminates or the date I receive this Release of
Claims. I also acknowledge that I am advised by the Company and its Affiliates
to seek the advice of an attorney prior to signing this Release of Claims; that
I have had sufficient time to consider this Release of Claims and to consult
with an attorney, if I wished to do so, or to consult with any other person of
my choosing before signing; and that I am signing this Release of Claims
voluntarily and with a full understanding of its terms.

     I further acknowledge that, in signing this Release of Claims, I have not
relied on any promises or representations, express or implied, that are not set
forth expressly in the Agreement. I understand that I may revoke this Release of
Claims at any time within seven (7) days of the date of my signing by written
notice to the General Counsel of the

<PAGE>

Company and that this Release of Claims will take effect only upon the
expiration of such seven-day revocation period and only if I have not timely
revoked it.

Intending to be legally bound, I have signed this Release of Claims under seal
as of the date written below.


Signature:
           -----------------------------
Name (please print):
                     -------------------
Date Signed:
             ---------------------------