<PAGE>
 
                                                                   Exhibit 10.52
                        EXECUTIVE SEPARATION AGREEMENT

     THIS EXECUTIVE SEPARATION AGREEMENT (the "Agreement") is entered into this
__ day of January, 1998, by and between Retix ("Company") and M.Y. Stephan
("Employee").


     WHEREAS, the Company and Employee have mutually agreed to terminate the
employment relationship and to establish a consulting arrangement between them.


     NOW, THEREFORE, IT IS AGREED as follows:


                     I.  Terms of Separation and Transition
                                     

     1.  Employee's resignation as President and Chief Executive Officer shall
be deemed effective as of the close of business on December 27, 1997 (the
"Termination Date").  Employee's resignation from any other positions with any
other entity which may be deemed to be an affiliate of the Company shall also be
effective on the Termination Date.


     2.  In contemplation of the separation, the parties agree:


         (a)    Employee's termination has arisen as a result of a "constructive
termination" under Section 5(b)(i) of Employee's Employment Agreement with the
Company dated September 27, 1995 (the "Employment Agreement") and Employee will
receive the payments, benefits and other severance provisions as provided in the
Employment Agreement and the confirming letter executed by Employee and the
members of the Compensation Committee of the Board of Directors of the Company
dated December 17, 1997 (the "Confirming Letter").  As a result, the Company and
Employee agree that Sections 1, 2 and 3 of the Addendum to Employment Agreement
dated December 17, 1997 shall not apply or remain operative.


         (b)    As indicated in the Confirming Letter, due to the consolidation
of operations between the Company and its Vertel Corporation subsidiary, under
the terms of Employee's Employment Agreement, all options previously granted to
Employee by the Company have become fully vested and all shares of the Company's
Common Stock that Employee has received upon exercise of these options have been
released from any rights of repurchase by the Company (other than related to the
pledging of such shares as security for the payment of any notes executed by
Employee in connection with their purchase).


         (c)    If Employee requests the release of any shares of the Company's
Common Stock held as security for the payment of outstanding promissory notes
owed to the Company by Employee in connection with the proposed sale or other
disposition of such shares by Employee, Employee agrees to repay at least that
percentage of the then outstanding principal owed under all such notes as is
equal to the percentage of shares then held by the Company as security as a
condition to the release of such shares.
<PAGE>
 
         (d)    To the extent provided by the Employment Agreement, the
Company's obligations to provide medical coverage or to reimburse for payments
made by Employee for continued medical coverage under the Consolidated Omnibus
Budget Reconciliation Act of 1985 ("COBRA") shall terminate at such time as
Employee becomes eligible to receive medical coverage providing a general level
of benefits similar to those offered by the Company to its employees generally
from another employer from which Employee is permitted to accept employment
under Section 7 of this Agreement.


         (e)    In addition to Employee's regular base compensation through the
Termination Date and the payments provided otherwise in this Agreement, Employee
shall be provided an additional severance payment of $255,000, which Employee
acknowledges has already been paid to him or deposited for his account in
accordance with his instructions, and shall pay an additional severance payment
of $120,000 on July 1, 1998.


         (f)    As contemplated by paragraph (2) of the Confirming Letter, the
Company shall retain Employee as a consultant as set forth in Article II below.


         (g)    Following the Termination Date, Employee shall remain a member
of the Board of Directors of the Company. As a member of the Board of Directors
of the Company who is not an employee of the Company following the Termination
Date, Employee will be eligible to receive options to purchase Common Stock of
the Company as and to the extent provided in the Company's 1996 Directors' Stock
Option Plan. The Company and Employee understand that the Board of Directors
will review continuing membership on the Board of Directors with respect to all
members as a result of the consolidation of operations of the Company with that
of its subsidiary Vertel Corporation and the search for additional Board
members, and that the Chief Executive Officer of the Company will provide his
recommendations in these regards. In all events, Employee's continued service as
a member of the Board of Directors shall be subject to the nomination of
Employee to such position and his election by the shareholders of the Company.


                          II.  CONSULTING RELATIONSHIP

                                        

     3.  Employee will make himself available at reasonable times and places to
perform consulting services for the Company during the period beginning on the
Termination Date and ending four years after the Termination Date (the
"Consulting Term").  The Consulting Term shall be unaffected by any possible
future termination in Employee's status as a member of the Board of Directors of
the Company.  Such services will include:


         (a)    continuing reasonable and customary transition support as
requested by the Company's current executive officers during the six months
following the Termination Date. Such support, generally in the form of
telephonic, fax or e-mail communication, shall be with respect to areas within
the general scope of Employee's duties while he was employed as an executive
officer by the Company, e.g. strategic and product planning and general business
direction and key account relationships in which Employee participated in a
meaningful way. No additional compensation shall be paid for such transition
support unless extraordinary in nature, in which case it shall be at Employee's
option to provide such extraordinary support and 
<PAGE>
 
for such additional compensation as may be agreed between the Company and
Employee.



         (b)    additional services concerning business strategy, market
development or such other subjects as may be agreed between the Company and
Employee. All consulting assignments and pertinent terms, including
compensation, shall be determined by Employee and the Chief Executive Officer or
Board of Directors of the Company.


     4.  Employee agrees that he will from time to time during the Consulting
Term keep the Company advised as to Employee's progress in performing any agreed
upon services and that Employee will, as requested by the Company, prepare
written reports with respect thereto.


     5.  The Company shall reimburse Employee for reasonable expenses incurred
on behalf of Company during the Consulting Term with respect to the provision of
consulting services, provided that such expenses are substantiated in accordance
with Company policies.


     6.  Employee understands and agrees that his obligations to the Company
under his existing Retix Employee Proprietary Information and Inventions
Agreement between Employee and Company dated August 1, 1995 (the "Proprietary
Information Agreement"), a copy of which is attached hereto as Exhibit A, shall
                                                               ---------       
continue through the Consulting Term and shall thereafter survive termination of
his relationship with Company under this Agreement.


     7.  Employee and the Company acknowledge and agree that Section 8
(Noncompetition Covenant) of the Employment Agreement shall continue to apply
through December 31, 1998 as a result of the payments Employee is receiving
pursuant to the Employment Agreement.  Employee may engage in other employment,
consulting or businesses other than as provided in such Section 8 provided such
activity does not preclude him from making himself available to provide periodic
consulting services to the Company.


     8.  Nothing in this Agreement shall in any way be construed to constitute
Consultant as an agent, employee or representative of the Company, and Employee
shall perform the consulting services contemplated in this Article II as an
independent contractor.

                            III.  RELEASE OF CLAIMS

     9.  Employee agrees that the foregoing provisions represent settlement in
full of all outstanding obligations owed to Employee by the Company.  Employee
and the Company, on behalf of themselves, and their respective heirs, executors,
officers, directors, employees, investors, shareholders, administrators,
predecessor and successor corporations,  and assigns, hereby fully and forever
release each other and their respective heirs, executors, officers, directors,
employees, investors, shareholders, administrators, predecessor and successor
corporations, and assigns, from, and agree not to sue concerning, any claim,
duty, obligation or cause of action relating to any matters of any kind, whether
presently known or unknown, suspected or unsuspected, that any of them may
possess arising from any omissions, acts or facts
<PAGE>
 
that have occurred up until and including the Effective Date of this Agreement
relating to Employee's employment with the Company including, without
limitation,


         (a)    any and all claims relating to or arising from Employee's
employment relationship with the Company and the termination of that
relationship;

         (b)    any and all claims relating to, or arising from, Employee's
right to purchase, or actual purchase of shares of stock of the Company;

         (c)    any and all claims for wrongful discharge of employment; breach
of contract, both express and implied; breach of a covenant of good faith and
fair dealing, both express and implied; negligent or intentional infliction of
emotional distress; negligent or intentional misrepresentation; negligent or
intentional interference with contract or prospective economic advantage; and
defamation;

         (d)    any and all claims for violation of any federal, state or
municipal statute, including, but not limited to, Title VII of the Civil Rights
Act of 1964, the Civil Rights Act of 1991, the Age Discrimination in Employment
Act of 1967, the Americans with Disabilities Act of 1990, the Fair Labor
Standards Act, and the California Fair Employment and Housing Act;

         (e)    any and all claims arising out of any other laws and regulations
relating to employment or employment discrimination; and

         (f)    any and all claims for attorneys' fee and costs.

         The Company and Employee agree that the release set forth in this
section shall be and remain in effect in all respects as a complete general
release as to the matters released. This release does not extend to any
obligations incurred under this Agreement.

     10. Employee acknowledges that he is waiving and releasing any rights he
may have under the Age Discrimination in Employment Act of 1967 ("ADEA") and
that this waiver and release is knowing and voluntary.  Employee and the Company
agree that this waiver and release does not apply to any rights or claims that
may arise under ADEA after the Effective Date of this Agreement.  Employee
acknowledges that the consideration given for this waiver and release Agreement
is in addition to anything of value to which Employee was already entitled.
Employee further acknowledges that he has been advised by this writing that:

         (a)    he should consult with an attorney prior to executing this
                                                   -----                  
Agreement;

         (b)    he has at least twenty-one (21) days within which to consider
this Agreement;

         (c)    he has at least seven (7) days following the execution of this
Agreement by the parties to revoke the Agreement; and

         (d)    this Agreement shall not be effective (the "Effective Date")
until the revocation period has expired.
<PAGE>
 
     11. The Company and the Employee represent that they are not aware of any
claim by either of them other than the claims that are released by this
Agreement. Employee and the Company acknowledge that they have had the
opportunity to consult with legal counsel concerning, and are familiar with, the
provisions of California Civil Code Section 1542, which provides as follows:

          A GENERAL RELEASE DOES NOT EXTEND TO CLAIMS WHICH THE CREDITOR DOES
          NOT KNOW OR SUSPECT TO EXIST IN HIS FAVOR AT THE TIME OF EXECUTING THE
          RELEASE, WHICH IF KNOWN BY HIM MUST HAVE MATERIALLY AFFECTED HIS
          SETTLEMENT WITH THE DEBTOR.

     Employee and the Company, being aware of said code section, agree to waive
expressly any rights they may have thereunder, as well as under any other
statute or common law principles of similar effect.

                         IV.  OTHER PROVISIONS; GENERAL

     12. Each party agrees to refrain from any disparagement, defamation,
slander of the other, or tortious interference with the contracts and
relationships of the other.

     13. Employee retains any rights to indemnification that he may have under
California law.  The Company will continue to be obligated to indemnify and
defend Employee to the full extent required by California Labor Code Section
2802 and any rights under the Company's Articles of Incorporation, Bylaws and
Indemnification Agreement between the Company and Employee dated as of August 1,
1995.

     14. This Agreement together with the written agreements referenced in this
Agreement represent the entire agreement and understanding between the Company
and Employee concerning Employee's separation from the Company, and together
supersede and replace any and all prior agreements and understanding concerning
Employee's relationship with the Company and his compensation by the Company.

     15. This Agreement may only be amended in writing signed by Employee and
the Chief Executive Officer of the Company.

     16. This Agreement shall be governed by the laws of the State of
California without reference to provisions concerning conflicts of laws.

     17. This Agreement is effective seven days after it has been signed by
both parties.

     18. This Agreement may be executed in counterparts, and each counterpart
shall have the same force and effect as an original and shall constitute an
effective, binding agreement on the part of each of the undersigned.
<PAGE>
 
     19. This Agreement is executed voluntarily and without any duress or undue
influence on the part or behalf of the parties hereto, with the full intent of
releasing all claims.  The parties acknowledge that:


          (a)  They have read this Agreement;

          (b) They have been represented in the preparation, negotiation, and
execution of this Agreement by legal counsel of their own choice or that they
have voluntarily declined to seek such counsel;

          (c) They understand the terms and consequences of this Agreement and
of the releases it contains;

          (d) They are fully aware of the legal and binding effect of this
Agreement.

     IN WITNESS THEREOF, the parties have executed this Agreement on the
respective dates set forth below.

                                    RETIX


Dated:  January ____, 1998        By _______________________________________
                                     Bruce Brown, President & CEO


                                     EMPLOYEE



Dated:  January ____, 1998
                                     _______________________________________
                                     M.Y. Stephan
<PAGE>
 
                                   EXHIBIT A
                                   ---------
                                        
                                        
                       PROPRIETARY INFORMATION AGREEMENT