<PAGE>
 
                                                                    EXHIBIT 10.4
                             EMPLOYMENT AGREEMENT


          This EMPLOYMENT AGREEMENT ("Agreement") is made and entered into this
1st day of January 1997 by and between Bay View Capital Corporation (the
"Corporation"), a Delaware Corporation, and John N. Buckley(the "Executive").

                                R E C I T A L S:

     A.   The parties desire to enter into this Agreement setting forth the
terms and conditions of the employment relationship between the Corporation and
the Executive.

     B.   The Board of Directors of the Corporation (or "Board") believes it is
in the best interests of the Corporation to enter into this Agreement with the
Executive in order to assure continuity of management of its wholly-owned
subsidiary, Bay View Federal Bank, a Federal Savings Bank ("Bank"), and has
approved and authorized the execution by the Corporation of this Agreement.

          NOW, THEREFORE, in consideration of the foregoing recitals and the
mutual promises herein contained, the parties hereto agree as follows:

          1.   Executive's Title, Duty, Authority and Time Commitment.
               ------------------------------------------------------ 

               a.   Title.  The Executive's position and title with the Bank
                    -----                                                   
shall be that of Senior Vice President, Director of Loan Administration.

               b.   Duties and Authority.  The Executive's duties and authority
                    --------------------                                       
shall be consistent with those of a Senior Vice President, Director of Loan
Administration. The Executive shall render such administrative and management
services to the Bank as are customarily performed by persons employed in the
banking and financial services industry in a similar executive capacity.  In
addition, the Executive shall perform such other duties as the Board, or its
authorized representative, may from time to time require.  All duties performed
by the Executive hereunder shall be in accordance with such reasonable standards
as are established from time to time by the Board, or its authorized
representative, and performance evaluations shall be conducted by or under the
direction of the Board and reviewed with the Executive annually.

               c.   Time Commitment.  During the "Term of this Agreement", as
                    ---------------                                          
defined in Section 5 of this Agreement, unless the Executive has obtained the
prior written consent of the Board, or its authorized representative,

                    (1) The Executive shall render his full productive time and
services to the Bank, keeping normal business hours, but is authorized to engage
in such banking trade

<PAGE>
 
association and related activities during normal business hours which, in his
judgment, with the concurrence of his immediate supervisor, are in the business
interests of the Bank; and

                    (2) The Executive shall not render personal services to any
other person or entity for compensation, either as an Executive, consultant,
director or officer, without first obtaining approval from the Chairman of the
Board. In no event shall such services conflict with the Executive's duty to the
Bank or adversely affect the Executive's judgment in the performance of his
responsibilities. Executive shall remain free to engage in community,
charitable, political and personal pursuits which do not interfere with his
performance under this Agreement.

          2.   Compensation.  The Corporation agrees that the Bank shall pay the
               ------------                                                     
Executive during the Term of this Agreement a base salary as follows: $150,000
per annum, with the salary to be reviewed on July 1, 1997, and annually on each
July 1st thereafter during the Term of this Agreement. Salary increases are not
guaranteed or automatic.  The salary provided for in this Agreement shall be
payable semi-monthly in accordance with the practices of the Bank.

          3.   Discretionary Bonuses.  The Executive may be entitled to
               ---------------------                                   
participate in discretionary bonuses and incentive payments to the extent
authorized and declared by the Board or its authorized representative.  For any
given calendar year, Executive shall be entitled to an annual performance bonus
in accordance with the terms of the incentive plan approved with respect to such
year by the Board or its authorized representative.

          4.   Additional Benefits.
               ------------------- 

               a.   Participation in Executive Benefit Plans.  The Executive
                    ----------------------------------------                
shall be entitled to participate in any plan of the Corporation, as such plan
may from time to time provide, relating to stock options, stock purchases,
pension, thrift, deferred profit-sharing, group insurance coverage, education or
retirement or other supplemental Executive benefits that the Corporation may
adopt for all of its Executives as a group.

               b.   Fringe Benefits.  The Executive shall be entitled to
                    ---------------                                     
participate in any other program which may be or become applicable to the
Corporation's executives, as such program may from time to time provide
including a reasonable expense account, the payment of reasonable expenses for
attending educational seminars and annual and periodic meetings of trade
associations, and other benefits which are commensurate with the
responsibilities and functions to be performed by the Executive under this
Agreement.

          5.   Term.
               ---- 

                                       2.
<PAGE>
 
               a.   Initial Term and Automatic Extension.  The initial term of
                    ------------------------------------                      
this Agreement shall be for a period of 12 months commencing on January 1, 1997
and ending on December 31, 1997 (the "Expiration Date").  On the Expiration Date
and on each anniversary date of the Expiration Date thereafter, the Term of this
Agreement shall be extended automatically by one additional year beyond the
then-current Expiration Date, unless either the Corporation or the Executive
gives contrary written notice to the other not less than 45 days in advance of
the date on which this Agreement would otherwise be extended. Reference herein
to the "Term of this Agreement" shall refer to the term as so extended.  Section
9, entitled "Change in Control," shall survive the expiration of this Agreement
for a period of one (1) year so long as Executive is employed by the
Corporation.

               b.   Consequences of Non-Extension.  If this Agreement is not
                    -----------------------------                           
extended, on the expiration of the Term of this Agreement, the Executive shall
be deemed to be employed by the Corporation for no specific term and the
Executive's rights as an Executive of the Bank shall be no less than those
provided by the laws of the State of California and the laws of the United
States; provided, however, that such post expiration employment may be
terminated at any time by the Executive or by the Board, or its authorized
representative, with or without cause by delivery to the Executive of a written
notice (the "Termination Notice") of such termination.

          6.   Voluntary Absences.  At such reasonable times as the Board, or
               ------------------                                            
its authorized representative, shall in its discretion permit, the Executive
shall be entitled, without loss of pay, to absent himself voluntarily from the
performance of his employment under this Agreement. All such voluntary absences
shall count as holidays, vacation time, floating holidays, sick leave, or other
paid time off as specified in Corporation policy, provided that:

               a.   Basic Vacation.  The Executive shall be entitled to an
                    ---------------                                        
annual paid vacation of 20 days per year.

               b.   Timing of Vacation.  Vacations shall be scheduled in a
                    ------------------                                    
reasonable manner by the Executive and subject to the Bank's needs, except that
the Executive shall take not less than ten (10) days vacation consecutively in
each calendar year.

          7.   Termination of Employment.
               ------------------------- 

               a.   Termination by Corporation.  The Executive's employment
                    --------------------------                             
under this Agreement may be terminated, with or without cause, at any time by
the Board, or its authorized representative, by delivery to the Executive of a
written notice (the "Termination Notice") of such termination.  The Termination
Notice shall state the effective date of such termination and whether such
termination 

                                       3.
<PAGE>
 
is for "cause," as defined in Section 7a(1), or without cause pursuant to
Section 7a(2). Unless the Termination Notice states that the termination is for
cause and states with reasonable particularity the cause, the termination shall
be deemed to be without cause pursuant to Section 7a(2). In the event an
arbitrator appointed pursuant to Section 13 of this Agreement determines that a
purported termination for cause was in fact without proper cause, the
termination shall nonetheless be effective, but the Executive shall be entitled
to the severance payment pursuant to Section 7a(2) hereof.

                    (1)  Termination by Corporation for Cause. The Executive's
                         ------------------------------------  
employment under this Agreement may be terminated at any time by the Board, or
its authorized representative, for "cause," which shall include, but not be
limited to the following:

                         (a) The commission by the Executive of an act of
misconduct (including, but not limited to, the violation of any law, rule,
regulation or cease and desist order applicable to the Executive or the
Corporation or its insured subsidiaries), or an act which constitutes a conflict
of interest with the Corporation or its stockholders, or a breach of a fiduciary
duty owed by the Executive;

                         (b) The Executive's breach of this Agreement,
dishonesty, incompetence, willful misconduct, habitual absence from work,
failure to perform duties, or negligence in the performance of stated duties;

                         (c) The Executive's becoming physically or mentally
incapable of performing the essential functions of his employment position, with
or without reasonable accommodation, for a period in excess of the applicable
leave entitlement mandated by Federal or State law; or

                         (d) Any criminal conviction of the Executive (other
than for a minor traffic violation or similar offense), whether or not in the
line of duty.

In the event of termination for cause under this Section 7a(1), the Executive
shall have no right to receive compensation or other benefits under this
Agreement for any period after such termination.

                    (2)  Termination by Corporation Without Cause. The
                         ----------------------------------------     
Executive's employment under this Agreement may be terminated at any time by the
Board, or its authorized representative, without cause; provided, however, that
unless the termination of this Agreement is for "cause," as set forth in Section
7a(1), or pursuant to Sections 7b, 7c, 7d, or 9, then, upon such termination, in
addition to any benefits that had accrued to the date of termination but in lieu
of any rights or benefits that would have 

                                       4.
<PAGE>
 
accrued following such termination, the Executive shall be entitled, upon
execution of a release acceptable to the Board, or its authorized
representative, to a lump sum severance payment of 12 months salary.

               b.   Termination by the Executive.  This Agreement and
                    ----------------------------                     
Executive's employment may be terminated by the Executive at any time upon 30
days written notice to the Corporation or upon such shorter period as may be
agreed upon between the Executive and the Board.

               c.   Termination by Death.  In the event of the death of the
                    --------------------                                   
Executive during the Term of this Agreement, the Executive's estate, or such
person as the Executive may have previously designated in writing for group
insurance purposes, shall be entitled to receive the salary due the Executive
through the last day of the calendar month in which his death shall have
occurred.

               d.   Suspension or Termination by Regulatory Action.
                    ---------------------------------------------- 

                         (1) Suspension. If the Executive is suspended from
                             ----------  
office and/or temporarily prohibited from participating in the conduct of the
affairs of any insured subsidiary of the Corporation by a notice served under
Section 8(e)(3) or (g)(1) of the Federal Deposit Insurance Act ("FDIA"), 12
U.S.C.(S)1818(e)(3); (g)(1), the Corporation, in its discretion, may suspend its
obligations under this Agreement as of the date of service, unless stayed by
appropriate proceedings. If the charges in the notice are dismissed, the
Corporation may in its discretion (i) pay the Executive all or part of the
compensation withheld while its obligations under this Agreement were suspended
and (ii) reinstate in whole or in part any of the obligations which were
suspended, but vested rights of the parties shall not be affected.

                         (2) Removal; Default; Supervisory Assistance or Merger.
                             -------------------------------------------------- 
The Corporation, in its discretion, may terminate all of its obligations under
this Agreement if: (A) the Executive is removed from office and/or permanently
prohibited from participating in the conduct of the affairs of any insured
subsidiary of the Corporation by an order issued under Section 8 (e)(4) or
(g)(1) of the FDIA, 12 U.S.C.(S)1818(e)(4); (g)(1); (B) any of the Corporation's
insured subsidiaries becomes in default (as defined in Section 3(x)(1) of the
FDIA, 12 U.S.C.(S)1813(x)(1)); (C) the Director of the Office of Thrift
Supervision ("OTS") or his or her designee at the time the Federal Deposit
Insurance Corporation or the Resolution Trust Corporation enters into an
agreement to provide assistance to or on behalf of any of the Corporation's
insured subsidiaries under the authority contained in Section 13(c) of the FDIA,
12 U.S.C.(S)1823(c); or (D) the Director of the OTS or his or her designee
approves a supervisory merger to resolve problems related to operation any of
the Corporation's insured 

                                       5.
<PAGE>
 
subsidiaries or when any of the Corporation's insured subsidiaries is determined
by the Director of the OTS to be in an unsafe or unsound condition; all as of
the effective date of the order, default, agreement to provide assistance or
approval of the merger, as the case may be, but vested rights of the parties
shall not be affected.

          8.   Disability.  If the Executive shall become disabled or
               ----------                                            
incapacitated to the extent that he is unable to perform the essential functions
of his employment position, with or without reasonable accommodation (as
determined in accordance with applicable law), he shall be entitled to receive
disability benefits of the type provided for other employees of the Corporation.
In such event, when the Executive has been absent 90 calendar days in excess of
utilized vacation and sick leave within a 120 calendar day period after the last
day the Executive worked, any rights of the Executive to receive the salary
provided in Section 2 of this Agreement shall be suspended until the Executive
is able to resume regular performance of his duties.

          9.   Change in Control.  If during the Term of this Agreement, or
               -----------------                                           
within one (1) year following the expiration of this Agreement and if Executive
is employed by the Corporation, there is a "change in control," as hereinafter
defined, the Executive shall be entitled, upon execution of a Severance
Agreement and Release as provided in Exhibit A, to a severance payment in the
event the Executive's employment is terminated, other than for "cause" as set
forth in Section 7a(1) or pursuant to Sections 7b (except as provided below), 7c
or 7(d), within twenty-four (24) months after the change in control.  The amount
of this payment shall equal two hundred percent (200%) of the Executive's annual
salary as of the date of termination and shall be in lieu of any severance
payment that would be due Executive under Section 7a(2). Such termination or
severance payment shall be in addition to all other amounts payable to the
Executive pursuant to this Agreement. This termination or severance payment
shall also be made in lieu of any severance payment that would be due Executive
under section 7b in the case of a termination of employment by the Executive
pursuant to Section 7b of this Agreement within twenty-four (24) months after a
change in control because during such twenty-four (24) month period there has
been a material diminution of or interference with the Executive's duties,
responsibilities and benefits as Senior Vice President, Director of Loan
Administration of the Bank. By way of example and not by way of limitation, any
of the following actions, if unreasonable or materially adverse to the
Executive, shall constitute such diminution or interference unless consented to
in writing by the Executive: (i) a change in the principal workplace of the
Executive to a location more than 25 miles from the Corporation's main office;
(ii) a reduction or adverse change in the salary or benefits which had
theretofore been provided to the Executive, other than as part of an overall
program 

                                       6.
<PAGE>
 
applied uniformly and with equitable effect to all members of senior management
of the Corporation or the Acquiror; or (iii) a change in Executive's title.

          Notwithstanding any other provision of this Agreement, if the value
and amounts of benefits under this Agreement, together with any other amounts
and the value of benefits received or to be received by the Executive in
connection with a change in control would cause any amount to be nondeductible
by the Corporation or the Acquiror for federal income tax purposes pursuant to
Section 280G of the Internal Revenue Code of 1986, as amended, then amounts and
benefits under this Agreement shall be reduced (not less than zero) to the
extent necessary so as to  maximize amounts and the value of benefits to the
Executive without causing any amount to become nondeductible by the Corporation
or the Acquiror pursuant to or by reason of such Section 280G.  The Executive
shall determine the allocation of such reduction among payments and benefits to
the Executive.

          The term "change in control" means (1) an event with respect to the
Acquiror of a nature that would be required to be reported in response to Item 1
of the current report on Form 8-K, as in effect on the date hereof, pursuant to
Section 13 or 15(d) of the Securities Exchange Act of 1934 (the "Exchange Act");
(2) any person (as the term is used in Sections 13(d) and 14(d) of the Exchange
Act)  is or becomes the beneficial owner (as defined in Rule 13d-3 under the
Exchange Act) directly or indirectly of securities of the Corporation or the
Acquiror representing 20% or more of the Corporation's or the Acquiror's
outstanding securities; or (3) a reorganization, merger, consolidation, sale of
all or substantially all of the assets of the Corporation or the Acquiror or a
similar transaction in which the Corporation or the Acquiror is not the
resulting entity.  The term "change in control" shall not include an acquisition
of securities by an employee benefit plan of the Corporation or the Acquiror.

          10.  Successors and Assigns.  The terms, provisions, covenants, and
               ----------------------                                        
conditions of this Agreement shall be binding upon and inure to the benefit of
the parties hereto and their respective successors and assigns; provided,
however, that the Executive may not sell, assign, pledge, hypothecate or
otherwise transfer this Agreement or any part thereof without the prior written
consent of the Corporation, which consent may be withheld by the Corporation for
any reason it deems appropriate.  "Successors and assigns" shall mean, in the
case of the Corporation, any successor pursuant to a merger, consolidation or
sale or transfer of all or substantially all of the assets of the Corporation.

          11.  Indemnification.  The Corporation hereby agrees that the
               ---------------                                         
Corporation shall indemnify the Executive to the fullest extent permitted by
the Corporation's Bylaws and by Delaware corporate law.

                                       7.
<PAGE>
 
          12.  Disclosure of Information.  Executive recognizes that as an
               -------------------------                                  
executive of the Corporation, Executive occupies a position of trust with
respect to much business information of a secret or confidential nature which is
the property of the Corporation and which will be imparted to Executive from
time to time in the course of Executive's duties.  Executive therefore agrees
that Executive shall not at any time, whether in the course of his employment or
thereafter, use or disclose directly or indirectly to any person outside the
Corporation any of such information, except as required in the ordinary course
of Executive's duties under this Agreement.

          13.  Arbitration.  The parties hereby agree that any controversy or
               -----------                                                   
dispute arising out of or relating to this Agreement shall be resolved pursuant
to this Section 13.

               a.   Agreement to Negotiate.  Prior to submitting any
                    ----------------------                          
controversy, dispute or claim arising out of, or relating to, this Agreement to
arbitration, the parties hereto agree to observe the following procedures:

                    (1) The party desiring to submit any such controversy,
dispute or claim to arbitration (the "claimant") first shall give written notice
thereof to the other party (the "recipient") setting forth in detail the
pertinent facts and circumstances relating to such controversy, dispute or
claim;

                    (2) The recipient shall have a period of fifteen (15) days
in which to consider the controversy, dispute or claim which is the subject of
the notice and to furnish in writing to the claimant a written statement of the
recipient's position;

                    (3) Within seven (7) days of claimant's receipt of
recipient's written statement, the parties shall meet in an effort to resolve
amicably any differences which may exist and, failing such resolution, either or
both of the parties shall have the right to submit the matter to arbitration.


               b.   Procedure for Arbitration.
                    ------------------------- 

                    (1) The parties hereby agree that to the extent allowable by
law any controversy, dispute or claim arising out of, or relating to, this
Agreement, or breach of this Agreement, including disputes concerning
termination of this Agreement as well as those based upon alleged tort or
unlawful acts including harassment or discrimination, shall be settled by
arbitration in San Mateo, California. The parties hereby waive any right to a
trial by court or jury. This agreement to arbitrate shall be specifically
enforceable. Judgment upon any award rendered by an arbitrator may be entered in
any court having jurisdiction.

                                       8.
<PAGE>
 
                    (2) Any demand for arbitration must be served on the other
party within forty-five (45) days of the act or omission giving rise to the
controversy, dispute or claim, except that the demand in writing may be made at
a later date in accordance with the applicable statute of limitation for any
statutory claim which allows a later date for the presentation of the claim.

                    (3) There shall be one impartial arbitrator chosen by the
Corporation from a list procured from the California Mediation and Conciliation
Service.

                    (4) The arbitrator shall not extend, modify or suspend any
of the terms of this Agreement.

                    (5) The decision of the Arbitrator within the scope of the
submission shall be final and binding on all parties, and any right to judicial
action on any matter subject to arbitration hereunder is hereby waived (unless
otherwise provided by applicable law), except suit to enforce this arbitration
award.

                    (6) Executive agrees that such arbitration shall be the
exclusive forum for any controversy, dispute or claim arising out of or relating
to this Agreement, or breach or termination of this Agreement. Executive further
expressly agrees that in arbitration his exclusive remedy shall be a money award
not to exceed the amount of wages he would have earned under this Agreement but
for the alleged violation and the Executive shall not be entitled to any other
remedy, at law or in equity, including but not limited to reinstatement, other
money damages, punitive damages and/or injunctive relief, unless required by
law.

                    (7) Each party shall pay such party's own attorney or other
representative, and the expenses of such party's witnesses and all other
expenses connected with his case unless otherwise required by law. Other costs
of the arbitration, including the cost of any record or transcript of the
arbitration, administrative fees, arbitrator's fees, and all other fees and
costs, shall be borne equally by the parties, unless otherwise required by law.

          14.  General Provisions.
               ------------------ 

               a.   Notices.  Any notice, request, demand or other communication
                    -------                                                     
required or permitted hereunder shall be deemed to be properly given when
personally served in writing and when deposited in the United States mail,
registered or certified, postage prepaid, addressed to the party at the last
address supplied to the sending party by the addressed party.

               b.   Waiver.  The waiver by any party of a breach of any
                    ------                                             
provision of this Agreement by the other shall not operate or 

                                       9.
<PAGE>
 
be construed as a waiver of any subsequent breach of the same provision or any
other provision of this Agreement.

               c.   Entire Agreement.  Except as provided herein, this Agreement
                    ----------------                                            
contains the entire agreement of the parties.  It supersedes any and all other
agreements, either oral or in writing, between the parties hereto with respect
to the employment of the Executive by the Corporation.  Each party to this
Agreement acknowledges that no representations, inducements, promises or
agreements, oral or otherwise, have been made by any party, or anyone acting on
behalf of any party, which are not embodied herein, and that no other agreement,
statement or promise not contained in this Agreement shall be valid and binding.

               d.   Amendments.  No amendments or additions to this Agreement
                    ----------                                               
shall be binding unless in writing and signed by both parties, except as herein
otherwise provided.

               e.   Paragraph Headings.  The paragraph headings used in this
                    ------------------                                      
Agreement are included solely for convenience and shall not affect, or be used
in connection with, the interpretation of this Agreement.

               f.   Severability.  The provisions of this Agreement shall be
                    ------------                                            
deemed severable and the invalidity or unenforceability of any provision shall
not affect the validity or enforceability of the other provisions hereof.

               g.   Governing Law.  Except where federal law governs, this
                    -------------                                         
Agreement is to be governed by and construed under the internal substantive laws
of the State of California unless otherwise specified in this Agreement (and not
under conflict of law principles) as such laws apply to contracts made and to be
performed entirely in the State of California.








          IN WITNESS WHEREOF, the parties have executed this Agreement on the
day and year first hereinabove written.

                         BAY VIEW CAPITAL CORPORATION,

                                      10.
<PAGE>
 
                         By: /s/ Edward H. Sondker
                            -----------------------------
                            Name:   Edward H. Sondker
                            Title:  President and
                                    Chief Executive Officer


                         THE EXECUTIVE



                           /s/ John F. Buckley
                          --------------------------------
                          John F. Buckley

                                      11.