<PAGE>
 
                                                                   EXHIBIT  10.3

                          CHANGE OF CONTROL SEVERANCE

          This Change of Control Severance Agreement ("Agreement") is entered
into as of January 22, 1996, by and between BSB BANCORP, INC. (the
"Corporation"), a Delaware corporation, and its wholly owned subsidiary BSB BANK
& TRUST COMPANY ("Employer"), a New York stock commercial bank, and ARTHUR C.
SMITH ("Executive").

                                 WITNESSETH:

          WHEREAS, Executive is currently employed by Employer as a Senior Vice
President; and

          WHEREAS, Employer desires to provide certain security to Executive in
connection with Executive's employment with Employer; and

          WHEREAS,   Executive and Employer desire to enter into this Agreement
pertaining to the terms of the security Employer is providing to Executive with
respect to his employment;

          NOW, THEREFORE, in consideration of the mutual covenants and promises
contained herein, and other good and valuable consideration, the receipt of
which is hereby acknowledged, the parties agree as follows:

          1. Term. The initial term of this Agreement shall be a period of three
             ----                                     
years from the date first above written, provided that the term of the Agreement
shall be extended automatically for an additional one year on each annual
anniversary date of this Agreement, unless either the Board of Directors of the
Employer or the Executive provide contrary written notice to the other not less
than 180 days in advance of any such anniversary date. In the event either the
Employer or the Executive provide such written notice, then the term hereof
shall not be extended, but the then current term shall continue for the period
remaining thereunder. Notwithstanding the foregoing, this Agreement shall
automatically expire and terminate on the Executive's normal retirement date at
age sixty-five (65) or on the Executive's early retirement under the Special
Service Retirement provision of the Employer's pension plan if the Executive
elects to take such early retirement.

          2. Benefits Upon Termination of Employment.
             --------------------------------------- 

          (a) If, at any time during the twelve-month period following a Change
in Control, (1) the employment of Executive with Employer is terminated by
Employer for any reason other than Good Cause, or (2) Executive terminates his
employment with Employer for Good Reason, Employer shall, during the Severance
Period, continue to pay Executive an amount equal to Executive's Base Salary.

          Such amount will be paid during the Severance Period in monthly or
other installments, similar to those being received by Executive at the date of
the Change in Control, and will commence as soon as practicable following the
date of termination of employment.  Executive shall receive any and all benefits
accrued under any Incentive Plans and Retirement Plans to the date of
termination of employment, the amount, form and time of payment of such benefits
to be determined by the terms of such Incentive Plans and Retirement Plans.

          (b) If upon the date of termination of Executive's employment,
Executive holds any options with respect to stock of Corporation, all such
options will immediately become exercisable upon such date and will be
exercisable for 90 days thereafter.  To the extent such acceleration of exercise
of such options is not permissible under the terms of any plan pursuant to which
the options were granted, Employer will pay to Executive, in a lump sum, within
90 days after termination of employment, an amount equal to the excess, if any,
of the aggregate fair market value of all stock of Corporation subject to such
options, determined on the date of termination of 
<PAGE>
 
employment, over the aggregate option price of such stock, and Executive will
surrender all such options unexercised. For the purposes of this Agreement, the
fair market value of the stock of Corporation shall be an amount equal to the
average of the bid and asked prices of the stock of the Corporation at the close
of the five business days preceding the date of Executive's termination of
employment.

          (c) During the Severance Period, Executive and his spouse will
continue to be covered by all Welfare Plans, maintained by Employer in which he
or his spouse were participating immediately prior to the date of his
termination as if he continued to be an employee of Employer; provided that, if
participation in any one or more of such Welfare Plans is not possible under the
terms thereof, Employer will provide substantially identical benefits.  If,
however, Executive obtains employment with another employer during the Severance
Period, such coverage shall be provided only to the extent that the coverage
exceeds the coverage of any substantially similar plans provided by his new
employer.

          (d) Notwithstanding any other provision of this Agreement or of any
other agreement, contract, or understanding heretofore or hereafter entered into
between Executive and the Corporation or Employer (or any subsidiary or
affiliate of either of them), except an agreement, contract, or understanding
hereafter entered into that expressly modifies or excludes application of this
paragraph (the "Other Agreements"), and notwithstanding any formal or informal
plan or other arrangement heretofore or hereafter adopted by the Corporation or
Employer (or any subsidiary or affiliate of either of them) for the direct or
indirect provision of compensation to Executive (including groups or classes of
participants or beneficiaries of which Executive is a member), whether or not
such compensation is deferred, is in cash, or is in the form of a benefit to or
for Executive (a "Benefit Plan"), Executive shall not have any right to receive
any payment or other benefit under this Agreement, any Other Agreement, or any
Benefit Plan if such payment or benefit, taking into account all other payments
or benefits to or for Executive under this Agreement, all Other Agreements, and
all Benefit Plans, would cause any payment to Executive under this Agreement to
be considered a "parachute payment" within the meaning of Section 28OG(b)(2) of
the Internal Revenue Code of 1986, as amended (a "Parachute Payment").  In the
event that the receipt of any such payment or benefit under this Agreement, any
Other Agreement, or any Benefit Plan would cause Executive to be considered to
have received a Parachute Payment under this Agreement, then Executive shall
have the right, in Executive's sole discretion, to designate those payments or
benefits under this Agreement, any Other Agreements, and/or any Benefit Plans,
which should be reduced or eliminated so as to avoid having the payment to
Executive under this Agreement be deemed to be a Parachute Payment.

          3. No Setoff. No payment or benefits payable to or with respect to
             ---------                                 
Executive pursuant to this Agreement shall be reduced by any amount Executive or
his spouse may earn or receive from employment with another employer or from any
other source, except as expressly provided in subsection 2(c).

          4. Death. If Executive dies during the Severance Period:
             -----                               

          (a) All amounts payable hereunder to Executive shall, during the
remainder of the Severance Period, be paid to his surviving spouse.

          (b) The spouse of Executive shall, during the remainder of the
Severance Period, be covered under all Welfare Plans made available by Employer
to Executive or his spouse immediately prior to the date of his death; provided
that, if participation in any one or more of such plans and arrangements is not
possible under the terms thereof, Employer will provide substantially identical
benefits.

          Any benefits payable under this Section 4 are in addition to any other
benefit due to Executive or his spouse or beneficiaries from Employer,
including, but not limited to, payments under any of the Incentive or Retirement
Plans.
<PAGE>
 
          5. Termination for Good Cause or Without Good Reason. If the
             -------------------------------------------------
employment of Executive with Employer is terminated (a) for any reason prior to
a Change in Control, or (b) after a Change in Control by Employer for Good
Cause, or by the voluntary action of Executive without Good Reason, Executive's
Base Salary (at the rate in effect on the date of termination) shall be paid
through the date of termination, and Employer shall have no further obligation
to Executive or his spouse under this Agreement, except for benefits accrued
under Incentive Plans pursuant to subsection 2 (a) above.

          6. Definitions.    For purposes of this Agreement:
             -----------                         

          (a) "Base Salary" shall mean the higher of Executive's annual base
salary at the rate in effect on the date of a Change in Control or the rate in
effect on the date of termination of employment.

          (b) "Change in Control" shall be deemed to have occurred if there has
been a change of control of a nature that would be required to be reported in
response to Item 6(e) of Schedule 14A of Regulation 14A promulgated under the
Securities Exchange Act of 1934, as amended ("Exchange Act"), or such item
thereof which may hereafter pertain to the same subject; provided that, and
notwithstanding the foregoing, a Change in Control shall be deemed to have
occurred if (i) any "person" (as that term is used in Sections (13 (d) and 14
(d) (2) of the Exchange Act) is or becomes the beneficial owner, directly or
indirectly, of securities of the Corporation or the Employer representing
twenty-five percent (25%) or more of the combined voting power of the
Corporation's then outstanding securities, or (ii) during any period of two (2)
consecutive years, individuals who at the beginning of such period constituted
the Boards of Directors of the Corporation and the Employer cease for any reason
to constitute at least a majority thereof unless the election of each Director,
who was not a Director at the beginning of the period, was approved by a vote of
at least two-thirds of the Directors then still in office who were Directors at
the beginning of the period, or (iii) the Corporation shall cease to be a
publicly owned corporation, or (iv) any merger or consolidation of the
Corporation with or into another entity shall occur as a result of which the
stockholders of the Corporation do not retain or acquire seventy-five (75%) or
more of the capital stock of the resulting entity.

     (c) "Good Cause" shall be deemed to exist if, and only if:
(1) Executive engages in acts or omissions constituting dishonesty, intentional
breach of fiduciary obligation or intentional wrongdoing or malfeasance; (2)
Executive is convicted of a criminal violation involving fraud or dishonesty; or
(3) Executive materially breaches the Agreement (other than by engaging in acts
or omissions enumerated in paragraphs (1) and (2) above), or materially fails to
satisfy the conditions and requirements of his employment with Employer, and
such breach or failure by its nature is incapable of being cured, or such breach
or failure remains uncured for more than 30 days following receipt by Executive
of written notice from Employer specifying the nature of the breach of this
paragraph (3), inattention by Executive to his duties shall be deemed a breach
or failure capable of cure.

     Without limiting the generality of the foregoing, the following shall not
constitute Good Cause: any personal or policy disagreement between Executive and
Employer or any member of the Board of Directors of Employer; or any action
taken by Executive in connection with his duties if Executive acted in good
faith and in a manner he reasonably believed to be in, and not opposed to, the
best interest of Employer and had no reasonable cause to believe his conduct was
unlawful.

     Notwithstanding anything herein to the contrary, in the event Employer
shall terminate the employment of Executive for Good Cause hereunder, Employer
shall give at least thirty (30) days prior written notice to Executive
specifying in detail the reason or reasons for Executive's termination.

     (d)  "Good Reason" shall exist if:
<PAGE>
 
     (1) there is a significant change in the nature or the scope of Executive's
authority;

     (2)  there is a reduction in Executive's rate of Base Salary;

     (3) Employer changes the principal location in which Executive is required
to perform services to one which is more than fifty miles from his current
location;

     (4) there is a reasonable determination by Executive that, as a result of a
change in circumstances significantly affecting his position, he is unable to
exercise the authority, powers, function or duties attached to his position;

     (e)  "Incentive Plans" shall mean any incentive, bonus deferred
compensation or similar plan or arrangement currently or hereafter made
available by Corporation or Employer in which Executive is eligible to
participate.

     (f) "Retirement Plans" shall mean any qualified or supplemental defined
benefit retirement plan or defined contribution retirement plan, currently or
hereinafter made available by Corporation or Employer in which Executive is
eligible to participate.

     (g) "Severance Period" shall mean the period beginning on the date the
Executive's employment with Employer terminates under circumstances described in
Section 2 and ending on the first to occur of: (1) the date 24 months
thereafter, or (2) the date Executive attains or would have attained age 65.

     (h) "Welfare Plan" shall mean any health and dental plan, disability plan,
survivor income plan and life insurance plan or arrangement currently or
hereafter made available by Employer in which Executive is eligible to
participate.

     7.   Benefits Unfunded.  All rights of Executive and his spouse or other
          -----------------                                                  
beneficiary under this Agreement shall at all times be entirely unfunded and no
provision shall at any time be made with respect to segregating any assets of
Corporation or Employer for payment of any amounts due hereunder.  Neither
Executive nor his spouse or other beneficiary shall have any interest in or
rights against any specific assets of Corporation or Employer, and Executive and
his spouse or other beneficiary shall have only the rights of a general
unsecured creditor of Employer.

     8.    Litigation Expense.  Employer shall pay Executive's out-of-pocket
           ------------------                                               
expenses, including attorney's fees, in connection with any judicial proceeding
to enforce this Agreement or to construe or determine the validity of this
Agreement or otherwise in connection herewith if the Executive is successful in
such litigation.

     9.    Applicable Law.  This Agreement shall be construed and interpreted
           --------------                                                    
pursuant to the laws of the State of New York.


     10.  Entire Agreement.  This Agreement contains the entire Agreement
          ----------------                                               
between Employer and Executive and supersedes any and all previous agreements,
written or oral, among the parties relating to his or her employment by the
Employer.  No amendment or modification of the terms of this Agreement shall be
binding upon the parties hereto unless reduced to writing and signed by Employer
and Executive.  This Agreement is the exclusive agreement between the
Corporation, Employer and the Executive regarding payments to the Executive in
the event of a change in control of the Corporation or the Employer.  During the
term of this Agreement, Executive shall not participate in or benefit from any
other change of control severance plan or policy which may be adopted 
<PAGE>
 
by the Corporation or Employer, provided that thereafter the Executive shall
participate in such plan or policy if one has been established by the
Corporation or Employer.

     11.  No Employment Contract.  Nothing contained in this Agreement shall be
          ----------------------                                               
construed to be an employment contract between Executive and Employer.
Executive is employed at will and Employer may terminate his employment at any
time, with or without cause.

     12.  Counterparts.  This Agreement may be executed in counterparts, each of
          ------------                                                          
which shall be deemed an original.

     13.  Severability.  In the event any provision of this Agreement is held
          ------------                                                       
illegal or invalid, the remaining provisions of this Agreement shall not be
affected thereby.

     14.  Successors.  This Agreement shall be binding upon and inure to the
          ----------                                                        
benefit of the parties hereto and their respective heirs, representatives and
successors.

     15.  Notice.  Notices required under this Agreement shall be in writing and
          ------                                                                
sent by registered mail, return receipt requested, to the following addresses or
to such other address as the party being notified may have previously furnished
to the others by written notice.

     If to Employer:                 Attention: William H. Rincker
     --------------                                               

     BSB Bank & Trust Company
     58-68 Exchange Street
     Binghamton, New York, 13902


     If to Executive:
     --------------- 

     Arthur C. Smith
     48 Margaret Street
     Johnson City, New York 13790
<PAGE>
 
     16.  Board Approval.  The obligations of Employer under this Agreement are
          --------------                                                       
contingent upon the approval or ratification by its Board of Directors of the
execution of this Agreement on its behalf.


     IN WITNESS WHEREOF, Executive has hereunto set his hand, and the
Corporation and the Employer have caused this agreement to be executed in their
name and on their behalf, all as of the day and year first above written.


                              BBB BANCORP, INC.
                              BSB BANK & TRUST COMPANY



                              By:   William H. Rincker
                                 ----------------------------------------
                                   William H. Rincker
                                   Chairman and Chief Executive Officer



                              By:   Arthur C. Smith
                                 ------------------------------------------
                                   Arthur C. Smith
                                   Senior Vice President