<PAGE>   1
                                                                    EXHIBIT 10.4

                              EMPLOYMENT AGREEMENT

        This Employment Agreement, dated this September 20, 2000 (this
"Agreement"), is entered into by and among Glacier Bancorp, Inc. ("Glacier"),
Western Security Bank ("Western Security") and Ralph K. Holliday (the
"Employee"). In consideration of Glacier's entering into the Plan and Agreement
of Merger with the parent company of Western Security, Westerfed Financial
Corporation ("Westerfed"), of even date herewith (the "Merger Agreement"), the
parties agree to enter into this Agreement in order to assure continuity of
management of Western Security and to foster the integration of Westerfed and
its subsidiaries with Glacier and its subsidiaries.

        1. Effective Date and Termination of Prior Employment Agreements. This
Agreement shall become effective on the Effective Date (as defined in the Merger
Agreement) and if the Merger Agreement is terminated, this Agreement shall
automatically terminate at the same time. Upon this Agreement becoming
effective, all employment, severance or change in control agreements between the
Employee and Westerfed and/or Western Security shall terminate, and as of such
time, the Employee shall be deemed to have waived his rights thereunder and to
have released Westerfed, Western Security, Glacier and its subsidiaries from any
claims thereunder.

        2. Employment. Glacier shall employ the Employee as a member of the
Glacier transition management team to integrate Westerfed and its subsidiaries
with Glacier and its subsidiaries, and Western Security shall employ the
Employee to continue to serve as President and Chief Executive Officer of
Western Security. While employed hereunder, the Employee shall report directly
to the Chief Executive Officer of Glacier.

        3. Term. The term of employment under this Agreement shall be the period
commencing on the Effective Date and concluding on the day before the first
anniversary of the Effective Date, unless Glacier extends such period by written
notice to the Employee not later than 10 months after the Effective Date (the
"Term"). In no event shall the Term extend past the date that is 18 months after
the Effective Date. The provisions of this Agreement shall survive the
expiration of the Term.

        4. Compensation.

            (a) Salary. Glacier or Western Security shall pay the Employee while
employed hereunder a salary of not less than $220,000 (per annum), at regular
intervals in accordance with Glacier's payroll practices for executives.

            (b) Bonuses and Incentive Compensation. The Employee shall be
eligible while employed hereunder for bonuses and incentive compensation on the
same basis as similarly situated executives of Glacier's financial institution
subsidiaries other than Western Security (the "Glacier Financial Subsidiaries").

<PAGE>   2

            (c) Employee Benefits. The Employee shall be eligible while employed
hereunder for the same health, welfare, employee, and qualified and nonqualified
plan benefits as, and on the same basis as, similarly situated executives of the
Glacier Financial Subsidiaries, with full credit for past service as provided
for in Section 6.3 of the Merger Agreement, except that:

               (i)    the Employee shall not be entitled to any options for
                      common stock of Glacier,

               (ii)   the amount of the Employee's annual paid vacation shall be
                      five weeks per year, and

               (iii)  Glacier or Western Security shall pay to the Employee an
                      auto allowance of $500 per month.

            (d) On the Effective Date, Glacier shall pay to the Employee in cash
a signing bonus of $75,000. The parties acknowledge that this payment is
separately bargained for consideration to induce the Employee to enter into this
Agreement and to agree to its terms and provisions.

        5. Termination of Employment. The term "Cause" shall mean termination of
the employment of the Employee by Glacier due to the Employee's personal
dishonesty associated with his employment, willful misconduct, breach of a
fiduciary duty involving personal profit, intentional failure to perform stated
duties, or willful violation of any law, rule, or regulation (other than traffic
violations or similar offenses) or final cease-and-desist order. The Employee
shall not be deemed to have been terminated for Cause unless and until there
shall have been delivered to the Employee a copy of a resolution, duly adopted
by the affirmative vote of not less than a majority of the entire membership of
the Glacier Board of Directors at a meeting of the Board called and held for
such purpose (after reasonable notice to the Employee and an opportunity for the
Employee, together with the Employee's counsel, to be heard before the Board),
stating that in the good faith opinion of the Board the Employee has engaged in
conduct described in the preceding sentence and specifying the particulars
thereof in detail.

        The term "Good Reason" shall mean (1) a change in the principal
workplace of the Employee to a location outside of a 30 mile radius from the
Western Security's headquarters office as of immediately prior to the Effective
Date; (2) any adverse change in the Employee's duties and responsibilities as an
officer of Western Security not including normal changes arising from shrinkage
in the size of Western Security or the number of Western Security personnel; and
(3) an adverse change in the Employee's salary, perquisites, benefits,
contingent benefits or vacation, other than as part of an overall program
applied uniformly and with equitable effect to all members of the senior
management of Glacier and the Glacier Financial Subsidiaries.

            (a) Termination Other Than Termination During the Term for Good
Reason or Involuntary Termination. In the event that the employment of the
Employee terminates at any time, including termination due to death or
disability, except as provided in Section 5(b) of this Agreement, Glacier or
Western Security shall pay him (or his estate) his accrued and unpaid salary and
the amount of any previously earned and unpaid bonuses and shall provide to him
(or

<PAGE>   3


his estate) his vested benefits under all health, welfare, employee, and
qualified and nonqualified plans.

            (b) Termination for Good Reason or Involuntary Termination During
the Term. If during the Term, either the Employee terminates his employment for
Good Reason or his employment is terminated involuntarily without Cause, Glacier
or Western Security shall pay him his salary and bonuses and shall provide to
him his benefits and vesting under all health, welfare, employee, and qualified
and nonqualified plans as if he had continued to be employed during the balance
of the Term.

            (c) Return of Property. When the Employee ceases to be employed by
Glacier and Western Security, he shall return all keys, passcards,
identification cards, all originals and copies whether in hard copy or
electronic or other form) of all documents, drawings, notes, memoranda, designs,
devices, diskettes, tapes, manuals and specifications which constitute
proprietary information or material of Glacier or Western Security and any other
property of Glacier or Western Security that are in his possession, including
without limitation any that are located in his place of work, his residence, his
car or in any other location under his control.

            (d) No Limitation on Right to Change of Control Payment. Nothing in
this Section 5 shall diminish the right of the Employee (or his estate) to
receive the Change of Control Payment (as defined below).

        6. Change of Control Payment. The term "Change of Control Payment" shall
mean an amount payable in a lump sum in cash equal to 299% of the Employee's
"base amount" as defined for purposes of Section 280G of the Internal Revenue
Code of 1986, as amended (with the last year of the base period to be calendar
year 2000), reduced to the extent necessary, if any, so as to maximize amounts
and the value of benefits to the Employee without causing any amount to become
nondeductible by Glacier (or members of the consolidated group of which Glacier
is a member) pursuant to or by reason of such Section 280G. The Employee shall
have the right to determine the allocation of such reduction among payments and
benefits to the Employee. The Employee hereby acknowledges that the Change of
Control Payment is a payment in connection with the change in ownership or
control of Westerfed and Western Security pursuant the Merger Agreement for
purposes of such Section 280G.

            (a) Entitlement to Change of Control Payment. The parties hereby
agree and acknowledge that the Employee is entitled to the Change of Control
Payment by virtue of the consummation of the merger contemplated by the Merger
Agreement and the Change of Control Payment shall be deemed earned but not
payable as of the Effective Date.

            (b) Time of Payment of Change of Control Payment. The Change of
Control Payment shall be paid by Glacier to the Employee (or his estate) on the
earlier of (i) the first anniversary of the Effective Date or (ii) the date on
which the Employee ceases to be employed by Glacier or Western Security for any
reason whatsoever.

        7. Covenants.

<PAGE>   4

            (a) Confidentiality. The Employee shall not, after the date of this
Agreement, including during and after the Term, use for his own purposes or
disclose to any other person or entity any confidential business information
concerning Western Security or Glacier or their business operations unless (i)
Western Security and/or Glacier, as applicable, consents to the use or
disclosure of its confidential business information, (ii) the use or disclosure
of such information is consistent with the Employee's duties under this
Agreement, or (iii) disclosure is required by law or court order. For purposes
of this Agreement, "confidential business information" includes, without
limitation, trade secrets (as defined in the Montana Uniform Trade Secrets Act,
Montana Code Section 30-14-402), various confidential information concerning all
aspects of current and future operations, nonpublic information on investment
management practices, marketing plans, pricing structure, and technology of
either Western Security or Glacier. The Employee shall also treat the terms of
this Agreement as confidential business information.

            (b) Nonsolicitation of Employees. During the year following the date
when the Employee ceases to be employed by Glacier and Western Security, the
Employee shall not persuade or entice, or attempt to persuade or entice, any
employee of Glacier or any of its subsidiaries to terminate his or her
employment.

            (c) Nonsolicitation of Customers and Others. During the year
following the date when the Employee ceases to be employed by Glacier and
Western Security, the Employee shall not persuade or entice, or attempt to
persuade or entice, any person or entity to terminate, cancel, rescind or revoke
his or its business or contractual relationships with Glacier or any of its
subsidiaries.

            (d) Noncompetition. During the year following the date when the
Employee ceases to be employed by Glacier and Western Security, the Employee
shall not provide management, supervisory or similar services in Montana to any
person or entity engaged in any business in Montana which competes with the
business of Glacier or Western Security in Montana as conducted as of the date
on which his employment terminates, including without limitation any preliminary
steps associated with the formation of a new bank. As consideration for this
covenant not to compete, Glacier shall pay to the Employee, in a single payment
on the date when his employment terminates, the following:

               (i)    if the Employee voluntarily terminates his employment
                      without Good Reason, $1,000,

               (ii)   if the Employee dies while employed or his employment is
                      terminated due to disability or for Cause, zero, or

               (iii)  if the Employee terminates his employment for Good Reason
                      or his employment is terminated by Glacier or Western
                      Security (other than due to disability or for Cause) (A)
                      $24,000 if his employment terminates before April 19,
                      2001, (B) $30,000 if his employment terminates on or after
                      April 19, 2001 but before April 19, 2002, or (C) $1,000 if
                      his employment terminates on or after April 19, 2002.

<PAGE>   5

            (e) Enforcement. The parties stipulate that, in light of all the
facts and circumstances of the relationship between the Employee on the one hand
and Glacier and Western Security on the other hand, the covenants and agreements
in this Section 7 are fair and reasonably necessary for the protection Glacier's
and Western Security's goodwill and other protectable interests. If a court of
competent jurisdiction declines to enforce any of such covenants and agreements
of the Employee not to compete with Glacier and Western Security, the parties
request the court to reform such provisions to restrict the Employee's ability
to compete with the Glacier and Western Security to the maximum extent the court
finds enforceable in time, scope of activities and geography. The Employee
hereby acknowledges that Glacier and Western Security will suffer immediate and
irreparable harm that will not be compensable by damages alone if he repudiates
or breaches any of the provisions of this Section 7 or threatens or attempts to
do so. Therefore, under such circumstances, Glacier and Western Security, in
addition to and without limitation of any other rights, remedies or damages
available to them at law or in equity, shall be entitled to obtain temporary,
preliminary and permanent injunctions in order to prevent or restrain any such
breach, and shall not be required to post a bond as a condition for the granting
of an injunction.

        8. No Mitigation. The Employee shall not be required to mitigate the
amount of any salary or other payment or benefit provided for in this Agreement
by seeking other employment or otherwise, nor shall the amount of any payment or
benefit provided for in this Agreement be reduced by any compensation earned by
the Employee as the result of employment by another employer, by retirement
benefits after the date when his employment by Glacier terminates or otherwise.

        9. Regulatory Provisions.

            (a) Temporary Suspension or Prohibition. If the Employee is
suspended and/or temporarily prohibited from participating in the conduct of
Western Security's affairs by a notice served under Section 8(e)(3) or (g)(1) of
the FDIA, 12 U.S.C. Section 1818(e)(3) and (g)(1), Western Security's
obligations under this Agreement shall be suspended as of the date of service,
unless stayed by appropriate proceedings. If the charges in the notice are
dismissed, Western Security may in its discretion (i) pay the Employee all or
part of the compensation withheld while its obligations under this Agreement
were suspended and (ii) reinstate in whole or in part any of its obligations
which were suspended.

            (b) Permanent Suspension or Prohibition. If the Employee is removed
and/or permanently prohibited from participating in the conduct of Western
Security's affairs by an order issued under Section 8(e)(4) or (g)(1) of the
FDIA, 12 U.S.C. Section 1818(e)(4) and (g)(1), all obligations of Western
Security under this Agreement shall terminate as of the effective date of the
order, but vested rights of the contracting parties shall not be affected.

            (c) Default of Western Security. If Western Security is in default
(as defined in Section 3(x)(1) of the FDIA), all obligations under this
Agreement shall terminate as of the date of default, but this provision shall
not affect any vested rights of the contracting parties.

            (d) Termination by Regulators. All obligations under this Agreement
shall be terminated, except to the extent determined that continuation of this
Agreement is necessary for

<PAGE>   6

the continued operation of Western Security: (i) by the Director of the Office
of Thrift Supervision (the "Director") or his or her designee, at the time the
Federal Deposit Insurance Corporation enters into an agreement to provide
assistance to or on behalf of Western Security under the authority contained in
Section 13(c) of the FDIA; or (ii) by the Director or his or her designee, at
the time the Director or his or her designee approves a supervisory merger to
resolve problems related to operation of Western Security or when it is
determined by the Director to be in an unsafe or unsound condition. Any rights
of the parties that have already vested, however, shall not be affected by any
such action.

            (e) Any payments made to the Employee pursuant to this Agreement, or
otherwise, are subject to and conditioned upon their compliance with 12 U.S.C.
1828(k) and any regulations promulgated thereunder.

        10. Notice. For the purposes of this Agreement, notices and all other
communications provided for in the Agreement shall be in writing and shall be
deemed to have been duly given when personally delivered or sent by certified
mail, return receipt requested, postage prepaid, to Glacier at its home office,
to the attention of the Chief Executive Officer, or, if to the Employee, to such
home or other address as the Employee has most recently provided in writing to.

        11. Amendments. No amendments or additions to this Agreement shall be
binding unless in writing and signed by all parties.

        12. Headings. The headings used in this Agreement are included solely
for convenience and shall not affect, or be used in connection with, the
interpretation of this Agreement.

        13. Arbitration. At the request of any party, the parties must submit
any dispute, controversy or claim arising out of or in connection with, or
relating to, this Agreement or any breach or alleged breach of this Agreement,
to arbitration under the American Arbitration Association's rules then in effect
(or under any other form of arbitration mutually acceptable to the parties). A
single arbitrator agreed on by the parties shall conduct the arbitration. If the
parties cannot agree on a single arbitrator, each party must select one
arbitrator and those two arbitrators shall select a third arbitrator. This third
arbitrator shall hear the dispute. The arbitrator's decision shall be final
(except as otherwise specifically provided by law) and bind the parties, and any
party may request any court having jurisdiction to enter a judgment and to
enforce the arbitrator's decision. The arbitrator shall provide the parties with
a written decision naming the substantially prevailing party or parties in the
action. This prevailing party or parties shall be entitled to reimbursement from
the other party or parties for its or their costs and expenses, including
reasonable attorneys' fees. All arbitration proceedings shall be held a place
designated by the arbitrator in the State of Montana. The arbitrator, in
rendering a decision as to any state law claims, shall apply Montana law.
Notwithstanding the foregoing provisions of this Section 13, if the Employee
violates Section 7 of this Agreement, Glacier and Western Security shall have
the right to initiate court proceedings to enforce Section 7 in lieu of an
arbitration proceeding.

<PAGE>   7

        14. Severability. The provisions of this Agreement shall be deemed
severable and the invalidity or unenforceability of any provision shall not
affect the validity or enforceability of the other provisions hereof.

        15. Governing Law. This Agreement shall be governed by the laws of the
United States to the extent applicable and otherwise by the laws of the State of
Montana.

        THIS AGREEMENT CONTAINS A BINDING ARBITRATION PROVISION WHICH MAY BE
ENFORCED BY THE PARTIES.

        IN WITNESS WHEREOF, the parties have executed this Agreement as of the
day and year first above written.

Attest:                                  GLACIER BANCORP, INC.

        /s/                                     /s/ Michael J. Blodnick
---------------------------------        ---------------------------------------
Secretary                                By:  Michael J. Blodnick
                                         Its:   President and CEO


Attest:                                  WESTERN SECURITY BANK

        /s/                                     /s/ James A. Salisbury
---------------------------------        ---------------------------------------
Secretary                                By: James A Salisbury
                                         Its:  Executive Vice President and CFO


                                         EMPLOYEE

                                                /s/ Ralph K. Holliday
                                         ---------------------------------------
                                         Ralph K. Holliday