<PAGE>

                                                                    EXHIBIT 10.4

                              EMPLOYMENT AGREEMENT

         This Agreement is made this 7th day of June, 2002, by and between
Southern Michigan Bancorp, Inc., a Michigan corporation, (the "Corporation")
and, John H. Castle (the "Employee").

         WHEREAS, the Board of Directors of the Corporation believes that the
future services of the Employee in the capacity of President and Chief Executive
Officer will be of great value to the Corporation;

         WHEREAS, the Corporation operates a wholly owned commercial banking
subsidiary known as Southern Michigan Bank & Trust which is engaged in the
general business of banking, hereinafter the "Bank"; and

         WHEREAS, the Employee is willing to serve in the employ of the
Corporation and the Bank on a full-time basis for the term of this Agreement.

         NOW, THEREFORE, in consideration of the premises and the mutual
covenants herein set forth, the parties hereto have agreed and do hereby
mutually agree as follows:

1.       Term -- Agreement to Serve

         The Corporation hereby employs for itself, the Bank and or any
         additional subsidiaries (hereinafter sometimes collectively referred to
         as the "Corporation"), the services of Employee for a period commencing
         as of the date first written above and terminating December 31, 2004
         (the "Termination Date"), subject to the rights of earlier termination
         hereinafter set forth, to perform the duties of President and Chief
         Executive Officer for the Corporation and the Bank. The Employee hereby
         accepts such employment in consideration of the compensation and the
         other terms and conditions herein provided, and agrees to serve the
         Corporation well and faithfully and to devote his best efforts to such
         employment as long as it shall continue hereunder. During the period of
         such employment, the Employee will devote all of his time and attention
         -- reasonable vacations, periods of illness and the like excepted -- to
         the affairs of the Corporation.

2.       Base Salary and Fringe Benefits

         Except as otherwise provided herein, as compensation for these services
         hereunder, the Corporation will pay to Employee, in installments and on
         dates in accordance with its normal payroll, during the period of his
         employment hereunder, a base salary at the aggregate rate of one
         hundred fifty thousand dollars ($150,000) per year, subject to the
         right of the parties, by mutual agreement, to adjust such rate upward
         in respect of any future calendar year or years after the date hereof,
         (hereinafter referred to as "Base Pay").

         In addition the Corporation shall:



                                      E-3
<PAGE>

         (a)      Provide four (4) weeks paid vacation annually.

         (b)      Reimburse fees and expenses incurred in connection with
                  business of the Corporation or the Bank including fees for
                  attendance at banking related conventions and similar items
                  approved by the Board of Directors.

         (c)      Provide an executive automobile and pay for the insurance,
                  maintenance and gasoline utilized by Employee.

         (d)      Provide a membership to Coldwater Country Club, and such other
                  clubs as agreed to by the parties from time to time, and pay
                  for all dues, fees and assessments.

3.       Bonus and Options

         (a)      Subject to the rules and regulations applicable thereto, the
                  Corporation shall provide for Employee's participation in any
                  option, incentive employee benefit plans or compensation
                  programs administered by the Corporation or the Bank or under
                  its direction, including any employee bonus plans as may
                  presently exist or are to be placed into effect after the date
                  hereof.

         (b)      Corporation shall negotiate in good faith an Incentive Bonus
                  Plan applicable to Employee and other key officers of the
                  Corporation, effective as of calendar year 2002.

4.       Termination of Employment

         The employment of the Employee under the terms of this Agreement shall
         cease and terminate as follows:

         (a)      Expiration of Term

                  On the Termination Date; or,

         (b)      Death

                  On the date of his death; or,

         (c)      Termination by the Corporation with Cause

                  For Cause at any time by action of the Board. For purposes
                  hereof, the term "Cause" shall mean removal by order of a
                  regulatory agency having jurisdiction over the Corporation or
                  the Bank, or the Employee's willful and repeated failure to
                  perform his duties under this Employment Agreement, which
                  failure has not been cured within thirty (30) days after the
                  Corporation gives notice thereof to the Employee; it being
                  expressly understood that negligence or bad judgment shall not
                  constitute "Cause" so long as such act or omission shall be
                  without intent of


                                      E-4
<PAGE>

                  personal profit and is reasonably believed by the Employee to
                  be in or not adverse to the best interests of the Corporation;
                  or,

         (d)      Disability

                  Upon receipt by the Employee of written notice from the
                  Corporation that, in its opinion, based on reliable medical
                  evidence, the Employee is unable by reason of permanent
                  physical or mental disability to continue the proper
                  performance of his duties hereunder. For purposes of this
                  Employment Agreement, the Employee's "permanent disability"
                  shall be deemed to have occurred after one hundred eighty
                  (180) consecutive days, during which one hundred eighty (180)
                  days the Employee, by reason of his physical or mental
                  disability or illness, shall have been unable to discharge his
                  duties under this Employment Agreement. The date of permanent
                  disability shall be such one hundred eightieth (180th) day. In
                  the event either the Corporation or the Employee, after
                  receipt of notice of the Employee's permanent disability from
                  the other, dispute that the Employee's permanent disability
                  shall have occurred, the Employee shall promptly submit to
                  physical examinations by three physicians in the Coldwater,
                  Michigan, area and, unless two of such physicians shall issue
                  their written statement to the effect that in their opinion,
                  based on their diagnosis, the Employee is capable of resuming
                  his employment and devoting his full time and energy to
                  discharging his duties within sixty (60) days after the date
                  of such statement, such permanent disability shall be deemed
                  to have occurred; or,

         (e)      Termination by the Corporation without Cause

                  At the election of the Corporation, at any time during the
                  term of this Agreement without cause.

         (f)      Termination in Connection with a Change in Control.

              i)  A "Change in Control" shall result if, and shall be deemed to
                  have occurred on the date of, a transaction pursuant to which:

                  (a) Any person or group (as such terms are used in connection
                      with Sections 13(d) and 14(d) of the Exchange Act) becomes
                      the "beneficial owner" (as defined in Rule 13(d)(3) and
                      13(d)(5) under the Exchange Act), directly or indirectly,
                      of securities of the Corporation representing 50% or more
                      of the combined voting power of the Corporation's then
                      outstanding securities;

                  (b) A merger, consolidation, sale of assets, reorganization,
                      or proxy contest is consummated and, as a consequence of
                      which, members of the Board in office immediately prior to
                      such transaction or event constitute less than a majority
                      of the Board thereafter;




                                      E-5
<PAGE>

                  (c) During any period of 24 consecutive months, individuals
                      who at the beginning of such period constitute the Board
                      (including for this purpose any new director whose
                      election or nomination for election by the Corporation's
                      stockholders was approved by a vote of at least one-half
                      of the directors then still in office who were directors
                      at the beginning of such period) cease for any reason to
                      constitute at least a majority of the Board; or

                  (d) A merger, consolidation or reorganization is consummated
                      with any other corporation pursuant to which the
                      shareholders of the Corporation immediately prior to the
                      merger, consolidation or reorganization do not immediately
                      thereafter directly or indirectly own more than fifty
                      percent (50%) of the combined voting power of the voting
                      securities entitled to vote in the election of directors
                      of the merged, consolidated or reorganized entity.

         Notwithstanding the foregoing, no trust department or designated
         fiduciary or other trustee of such trust department of the Corporation
         or a subsidiary of the Corporation, or other similar fiduciary capacity
         of the Corporation with direct voting control of the stock shall be
         treated as a person or group within the meaning of subsection (i)(a)
         hereof. Further, no profit-sharing, employee stock ownership, employee
         stock purchase and savings, employee pension, or other employee benefit
         plan of the Corporation or any of its subsidiaries, and no trustee of
         any such plan in its capacity as such trustee, shall be treated as a
         person or group within the meaning of subsection (i)(a) hereof.

         ii)  If during the term of this Agreement and after the date of a
              Change in Control, Employee is discharged without Cause or
              Employee resigns because he has: (1) been demoted, (2) had his
              compensation reduced, (3) had his principal place of employment
              transferred away from Branch County, Michigan or a county
              contiguous thereto, or (4) had his job title, status or
              responsibility materially reduced, then the Corporation shall make
              the payments to Employee set forth in subsection (iv) of this
              Section (f).

         iii) If Employee is discharged by the Corporation other than for Cause
              and there is a Change in Control within twelve (12) months
              following the discharge, then the Corporation shall make the
              payments to Employee set forth in subsection (iv) of this Section
              (f).

         iv)  In the event of the termination of Employee's employment as
              described in subSection (ii) or (iii) above, Employee shall be
              entitled to receive: (1) a cash payment equal to 2.99 times his
              Compensation (as defined below), or (2) upon Employee's election,
              2.99 times his Compensation payable in equal monthly payments, in
              cash, without interest. The lump sum cash payment or the first
              monthly cash payment, as the case may be, shall be paid at the end
              of the first month commencing after the Employee's termination of
              employment in the case of a benefit entitlement under subsection
              (ii) or (iii) above, and in the event of the election by Employee
              to receive monthly payments, shall continue each consecutive month
              thereafter until 36 payments have been made; provided that more
              than 90 days before a Change in Control, Employee


                                      E-6
<PAGE>

              may elect to defer the commencement date of payments (whether lump
              sum or monthly) by a period of three years, in which case interest
              shall accrue on unpaid balances at 120% of the applicable federal
              rate through the date of the payment of such lump sum, or
              commencement of payments, as the case may be.

              In the event Employee dies before collecting all amounts and
              benefits due under this Section, any payments owing shall be paid
              to the person or persons as stated in the last designation of
              beneficiary concerning this Agreement signed by Employee and filed
              with the Corporation, and if not, then to the personal
              representative of the Employee.

              The payments and benefits provided for in this Section 4(f) are in
              lieu of compensation, benefits or amounts the Employee might
              otherwise be entitled to under the Corporation's severance policy
              or otherwise payable by the Corporation by reason of termination
              of employment.

              The term "Compensation," as used in this section, shall mean the
              average of the sum of Employee's base salary plus any cash bonuses
              for the last three complete calendar years preceding Employee's
              termination of employment. Compensation shall not include any
              amount, other than base salary and cash bonuses, included in
              Employee's taxable compensation for federal income tax purposes
              and reported to Employee and the Internal Revenue Service ("IRS"),
              such as the reporting of previously deferred compensation or gain
              realized upon exercise of any non qualified stock options.

         v)   In the event the payments required under this Agreement, when
              added together with any other amounts required to be included by
              Employee under the provisions of the Internal Revenue Code of
              1986, as amended, result in an "Excess Parachute Payment," as that
              term is defined in Section 280G of the Code, then the amount of
              the payments provided for in this agreement shall be reduced to
              that amount which causes no excise tax to be imposed under Section
              4999 (or any successor thereto) of the Code.

         vi)  Any subsequent employment by Employee shall not reduce the
              obligation of the Corporation to make the full payments and
              provide the full benefits specified herein and Employee shall have
              no obligation to seek other employment or otherwise mitigate the
              effect of his discharge from employment.

         (g)      Payments Upon Termination other than Change in Control.

                  Upon termination of employment of the Employee pursuant to
                  Section 4(b),(c) or (d) above, the Employee shall be entitled
                  to receive the amount of Base Pay and Benefits provided for in
                  Paragraphs 2 and 3 hereof through the date of his termination
                  of employment. In the event of the termination of employment
                  of Employee pursuant to Sections 4(a) or (e) above, the
                  Corporation shall pay to the Employee his Base Pay and other
                  benefits listed in Paragraphs 2 and 3 until the later of
                  December 31, 2004 or One (1) year after the Termination Date.



                                      E-7
<PAGE>

5.       Right to Other Benefits.

         Except as otherwise specified herein, nothing in this Agreement shall
         abridge, eliminate, or cause Employee to lose Employee's right or
         entitlement to any other Corporation benefit to which Employee may be
         entitled due to his status as an employee under any plan or policy of
         Corporation on such terms and conditions as are required of any
         employee under any plan or policy of Corporation. Further, nothing in
         this Agreement shall create in Employee any greater rights or
         entitlements, except as specified in this Agreement. The plans and
         policies referred to in this Paragraph 5 include, but are not limited
         to, life insurance plans, dental, disability or health insurance
         benefits, severance policies, club memberships, and accrued vacation
         pay.

6.       Noncompetition and Nonsolicitation Agreement and Business Protection.

         Notwithstanding anything to the contrary contained elsewhere in this
         Agreement:

         (a)      Noncompetition Agreement and Nonsolicitation Agreement

              (i) In view of Employee's importance to the success of the
                  Corporation, Employee and Corporation agree that the
                  Corporation would likely suffer significant harm from
                  Employee's competing with Corporation during Employee's term
                  of employment with Corporation and for some period of time
                  thereafter. Accordingly, Employee agrees that Employee shall
                  not engage in competitive activities while employed by
                  Corporation and during the Restricted Period. Employee shall
                  be deemed to engage in competitive activities if he shall,
                  without the prior written consent of the Corporation, (i) in
                  Branch County, Michigan, and counties contiguous thereto
                  (including the municipalities therein), render services
                  directly or indirectly, as an employee, officer, director,
                  consultant, advisor, partner or otherwise, for any
                  organization or enterprise which competes directly or
                  indirectly with the business of Corporation or any of its
                  affiliates in providing financial products or services
                  (including, without limitation, banking, insurance, or
                  securities products or services) to consumers and businesses,
                  or (ii) directly or indirectly acquires any financial or
                  beneficial interest in (except as provided in the next
                  sentence) any organization which conducts or is otherwise
                  engaged in a business or enterprise in Branch County,
                  Michigan, and counties contiguous thereto (including all
                  municipalities) which competes directly or indirectly with the
                  business of Corporation or any of its affiliates in providing
                  financial products or services (including, without limitation,
                  banking, insurance or securities products or services) to
                  consumers and businesses. Notwithstanding the preceding
                  sentence, Employee shall not be prohibited from owning less
                  that 1 percent of any publicly traded corporation, whether or
                  not such corporation is in competition with Corporation. For
                  purposes of this Paragraph 6 the term "Restricted Period"
                  shall equal One (1) year, commencing as of the date of
                  Employee's termination.




                                      E-8
<PAGE>

            (ii)  While employed by Corporation and for a period of One (1) year
                  following Employee's termination of employment with
                  Corporation, Employee agrees that Employee shall not, in any
                  manner directly (i) solicit by mail, by telephone, by personal
                  meeting, or by any other means, any customer or prospective
                  customer of Corporation to whom Employee provided services, or
                  for whom Employee transacted business, or whose identity
                  become known to Employee in connection with Employee's
                  services to Corporation (including employment with or services
                  to any predecessor or successor entities), to transact
                  business with a person or an entity other than the Corporation
                  or its affiliates or reduce or refrain from doing any business
                  with the Corporation or its affiliates or (ii) interfere with
                  or damage (or attempt to interfere with or damage) any
                  relationship between Corporation or its affiliates and any
                  such customer or prospective customer. The term "solicit" as
                  used in this Agreement means any communication of any kind
                  whatsoever, inviting, encouraging or requesting any person to
                  take or refrain from taking any action with respect to the
                  business of Corporation and its subsidiaries.

            (iii) While employed by Corporation and for a period of One (1) year
                  following Employee's termination of employment with
                  Corporation, Employee agrees that Employee shall not, in any
                  manner directly solicit any person who is an employee of
                  Corporation or any of its affiliates to apply for or accept
                  employment or a business opportunity with any other person or
                  entity.

            (iv)  The parties agree that nothing herein shall be construed to
                  limit or negate the common law of torts or trade secrets where
                  it provides broader protection than that provided herein.

           (v)    Notwithstanding the foregoing, this Section 6(a) shall not
                  apply in the event of termination pursuant to Section 4(f).

         (b).     Confidential Information

         Employee has obtained and may obtain confidential information
         concerning the businesses, operations, financial affairs,
         organizational and personnel matters, policies, procedures and other
         non-public matters of Corporation and its affiliates, and those of
         third-parties that is not generally disclosed to persons not employed
         by Corporation or its subsidiaries. Such information (referred to
         herein as the "Confidential Information") may have been or may be
         provided in written form or orally. Employee shall not disclose to any
         other person the Confidential Information at any time during his
         employment with Corporation or after the termination of his employment,
         provided that Employee may disclose such Confidential Information only
         to a person who is then a director, officer, employee, partner,
         attorney or agent of Corporation who, in Employee's reasonable good
         faith judgment, has a need to know the Confidential Information.




                                      E-9
<PAGE>

         (c)      Remedies

           (i)    Employee acknowledges that a violation on Employee's part of
                  this Paragraph 6 would cause immeasurable and irreparable
                  damage to Corporation. Accordingly, Employee agrees that
                  notwithstanding Paragraph 13 hereof, Corporation shall be
                  entitled to injunctive relief in any court of competent
                  jurisdiction for any actual or threatened violation of any of
                  the provisions of this Paragraph 6, in addition to any other
                  remedies it may have.

           (ii)   In addition to Corporation's right to seek injunctive relief
                  as set forth in subsection (i) above of this Section 6(c) in
                  the event that Employee shall violate the terms and conditions
                  of this Paragraph 6, Corporation may: (i) make a general claim
                  for damages and (ii) terminate any payments or benefits
                  payable by Corporation, if applicable, to Employee.


7.       Inventions, Discoveries and Improvements

         The Employee hereby agrees to assign and transfer to the Corporation,
         its successors and assigns, his entire right, title and interest in and
         to any and all inventions, discoveries, trade secrets and improvements
         thereto which he may discover or develop, either solely or jointly with
         others, during his employment hereunder and for a period of one year
         after termination of such employment, which would relate in any way to
         the business of the Corporation or any parent, subsidiary or affiliate
         of the Corporation, together with all rights to letters patent,
         copyrights or trademarks which may be granted with respect thereto.
         Immediately upon making or developing any invention, discovery, trade
         secret or improvement thereto, Employee shall notify the Corporation
         thereof and shall execute and deliver to the Corporation, without
         further compensation, such documents as may be necessary to assign and
         transfer to the Corporation his entire right, title and interest in and
         to such invention, discovery, trade secret or improvement thereto, and
         to prepare or prosecute applications for letters patent with respect to
         the same in the name of the Corporation.

8.       Confidential Information

         Employee shall not at any time, in any manner, while employed by the
         Corporation or thereafter, either directly or indirectly, except in the
         course of carrying out the Corporation's business or as previously
         authorized in writing on behalf of the Corporation, disclose or
         communicate to any person, firm, or corporation, any information of any
         kind concerning any matters affecting or relating to the Corporation's
         business or any of its data, figures, projections, estimates, customer
         lists, tax records, personnel histories, and accounting procedures of
         the Corporation, without regard to whether any or all of such
         information would otherwise be deemed confidential or material.




                                      E-10
<PAGE>

9        Non-Assignability

         (a)      Neither party to this Agreement shall have the right to assign
                  this Agreement or any rights or obligations hereunder provided
                  that nothing herein shall prevent the Employee from
                  designating one or more members of his family or a trust or
                  trusts for the members of his family as a beneficiary or
                  beneficiaries entitled to receive payments hereunder as
                  heretofore specified.

         (b)      Except as provided above, no title to any payments which shall
                  become due and payable to the Employee, his personal
                  representative or designated beneficiary under the provisions
                  hereof, shall be vested in him or any of them until the actual
                  payment thereof is made to such person by the Corporation in
                  accordance with the provisions of this Agreement. Neither he
                  nor any of them shall have the right or power to transfer,
                  assign, anticipate or encumber any interest in any such
                  payment, prior to the actual receipt thereof from the
                  Corporation. Neither this Agreement, the Corporation nor any
                  person's rights hereunder shall be liable for the debt,
                  contract or engagement of any of them. None of them shall be
                  permitted to appoint any agent or attorney-in-fact and except
                  as provided herein, to collect or receive his share of such
                  payments or any part thereof unless permission to do so shall
                  be specifically granted by the Corporation in writing. The
                  Corporation, in the absence of such written permission, shall
                  not in any manner recognize such appointment, transfer,
                  assignment or encumbrance.

         (c)      If the Employee or any personal representative or any
                  designated beneficiary attempts to transfer, assign or
                  encumber his interest in such payments, or any part thereof,
                  prior to the payment or distribution thereof to him or her;
                  or, if any transfer or seizure thereof is attempted to be made
                  or brought through the operation of any bankruptcy or
                  insolvency law, the right of the person taking such action or
                  concerned therein or affected thereby, and who would, but for
                  this provision, be entitled to receive such payments, or any
                  part thereof, shall forthwith and ipso facto terminate, all
                  rights bestowed on any such person being hereby, on the
                  happening of any such event, expressly revoked; and the
                  Corporation shall thereafter, in its absolute discretion, at
                  such time or times as it deems proper, cause such part of such
                  person's theretofore existing share of such payments to be
                  paid to such person or persons, including the Employee, of any
                  parent, spouse or child of said person, as the Corporation, in
                  its uncontrolled discretion, shall deem advisable; and the
                  remainder of such payments, if any, may be distributed by the
                  Corporation to the person or person who would have been
                  entitled to receive the same if such person had died
                  immediately prior to said attempted transfer, assignment or
                  encumbrance, or attempted transfer or seizure by operation of
                  law.

10.      Binding Effect

         This Agreement shall be binding upon and inure to the benefit of any
         successor of the Corporation, and any such successor shall be deemed
         substituted for the Corporation


                                      E-11
<PAGE>

         under the terms of this Agreement. As used in this Agreement, the term
         "successor" shall include any person, firm, corporation, or other
         business entity which, at any time, whether by merger, purchase, or
         otherwise, acquires all or substantially all of the assets or business
         of the Corporation.

11.      Entire Agreement

         This Agreement contains the entire agreement of the parties hereto
         concerning the subject matter hereof, and cancels any and all other
         oral or written agreements or understandings between the parties with
         respect to the subject matter hereof, provided, however, that the
         provisions of the Second Addendum, other than paragraph 4 thereof,
         shall remain in full force and effect. The Agreement may not be changed
         orally, but only by agreement in writing signed by both parties.

12.      Authorization for Acts of Corporation

         Any act, request, approval, consent or opinion of the Corporation
         hereunder shall be authorized, given or expressed by resolution of its
         Board of Directors.

13.      Arbitration.

         Subject to the Corporation's right to seek injunctive relief under
         subsection 6(c)(i) of this Agreement, the parties hereto agree to
         arbitrate any issue, misunderstanding, disagreement or dispute in
         connection with the terms in effect in this Agreement in accordance
         with the Rules of the American Arbitration Association, before one
         arbitrator mutually agreeable to the parties. If either party
         determines that the parties have been unable to agree upon one
         arbitrator, then such party may appoint one arbitrator and require the
         other party to appoint a second arbitrator. Whereupon, the two
         appointed arbitrators shall appoint a third neutral arbitrator. If the
         arbitrators selected by the parties are unable or fail to agree upon
         the third arbitrator, the American Arbitration Association shall select
         the third arbitrator. Failure by a party to either (i) accept as
         mutually agreeable, or (ii) appoint an arbitrator, within 30 days of
         receipt of notice of the appointment of an arbitrator by the other
         party, shall be deemed as acceptance of arbitration by such single
         arbitrator. The arbitration shall occur in Coldwater, Michigan, or such
         other place as mutually agreed upon. The prevailing party shall be
         entitled to recover any and all costs associated with any arbitration
         proceeding (and any subsequent proceeding to enforce rights thereunder)
         including the recovery of reasonable attorneys fees. Judgement on the
         award rendered by the arbitrator(s) may be entered in any court having
         jurisdiction thereof.

14.      Governing Law

         This Agreement is executed and delivered in the State of Michigan and
         is intended to be interpreted, construed and enforced in accordance
         with the laws of such State.




                                      E-12
<PAGE>

         IN WITNESS WHEREOF, the Corporation has caused this Agreement to be
executed on its behalf by the Chairman of its Board of Directors, and the
Employee has signed this Agreement, all as of the date and year first above
written.


                                      Southern Michigan Bancorp, Inc.



                                      By: /s/James Morrison
                                          -----------------------------------
                                            James Morrison,
                                                   Chairman, Board of Directors


                                          /s/ John H. Castle
                                          -----------------------------------
                                            John H. Castle, Employee





                                      E-13