<PAGE>
 
                                                                    Exhibit 10.5

                            SEVERANCE PAY AGREEMENT
                            -----------------------


     THIS SEVERANCE PAY AGREEMENT ("Agreement") is made and entered into as of
the 1st day of July, 1998, by and among PREMIER BANK ("Premier"), THE BANK OF
GWINNETT COUNTY ("Gwinnett") (Premier and Gwinnett shall sometimes be
collectively referred to as "Employers"), and LINDA S. GEORGE ("Employee") and
shall become effective only upon the closing of that certain Agreement and Plan
of Reorganization, dated as of February 5, 1998, by and between Button Gwinnett
Financial Corporation ("Button Gwinnett") and the Holding Company (the
"Effective Date").

                              W I T N E S S E T H:

     WHEREAS, Premier is a wholly owned subsidiary of  Premier Bancshares, Inc.
(the "Holding Company");

     WHEREAS, as of the Effective Date, Button Gwinnett, a Georgia corporation,
merged with and into the Holding Company, and Gwinnett became a wholly-owned
banking subsidiary of the Holding Company;

     WHEREAS, as of the Effective Date, Employee became Vice President of
Construction Lending of Premier and continued as Vice President of Construction
Lending of Gwinnett;

     WHEREAS, the Boards of Directors of Employers consider the establishment
and maintenance of highly competent and skilled management personnel for
Employers to be essential to protecting and enhancing the best interests of
Employers;

     WHEREAS, the Boards of Directors of Employers are desirous of inducing
Employee to remain in the employ of Employers, subject to the terms and
conditions hereof; and

     WHEREAS, the parties agree that the provisions of this Agreement shall
control with respect to the rights and obligations of the parties resulting from
the future termination of the employment of Employee by Employers;
<PAGE>
 
     NOW, THEREFORE, for and in consideration of the mutual covenants herein
contained, and other good and valuable consideration, the receipt and
sufficiency of which are hereby acknowledged, the parties hereto do hereby agree
as follows:

     1.  Definition of Change of Control.  "Change of Control" shall be deemed
         -------------------------------                                      
to have occurred:

          (a) Upon the consummation of any transaction in which any person (or
     persons acting in concert), partnership, corporation, or other organization
     shall own, control, or hold with the power to vote more than fifty percent
     (50%) of any class of voting securities of the Holding Company;
      --                                                            

          (b) Upon the consummation of any transaction in which the Holding
     Company, or substantially all of the assets of the Holding Company, shall
     be sold or transferred to, or consolidated or merged with, another
     corporation; or

          (c) Upon the consummation of any transaction in which either or both
     Employers, or substantially all of the assets of either or both Employers,
     shall be sold or transferred to, or consolidated or merged with, another
     corporation which is not a majority owned subsidiary of the Holding
     Company;

provided, however, if either or both Premier and Gwinnett shall become a
subsidiary of another corporation or shall be merged or consolidated into
another corporation and a majority of the outstanding voting shares of the new
parent or surviving corporation are owned, immediately after such acquisition,
merger, or consolidation, by the owners of a majority of the voting shares of
the Holding Company immediately before such acquisition, merger, or
consolidation, then no Change of Control shall be deemed to have occurred.  It
is contemplated by the parties that, during the Term of Employment, Gwinnett may
be merged with and into Premier; and further that, upon such merger, Premier,
for all purposes under this Agreement, shall be deemed the sole Employer of
Employee and referred to as "Employers."

                                      -2-
<PAGE>
 
     2.   Employment.  Employee agrees to remain in the employ of Employers as
          ----------                                                          
an employee at will under Georgia law.  Employee agrees to perform faithfully
such services as are reasonably consistent with her position(s) and shall from
time to time be assigned to her by the Boards of Directors of Employers in a
trustworthy and businesslike manner for the purpose of advancing the interests
of  Employers.  The Boards of Directors of Employer may also from time to time
change Employee's position or alter her duties and responsibilities and assign a
new position or new duties and responsibilities that are similar in scope and
nature to Employee's existing position, duties, and responsibilities without
invalidating this Agreement or effecting the termination of Employee.  At all
times, Employee shall manage and conduct the business of Employers in accordance
with the policies established by the Boards of Directors of Employers and in
compliance with applicable regulations promulgated by governing regulatory
agencies.  Responsibility for the supervision of Employee shall rest with the
Chief Executive Officer(s) of Employers.  The Boards of Directors of Employers
shall have the authority to terminate Employee, subject only to the provisions
outlined in paragraph 4 of this Agreement.

     3.   Performance of Duties.  During the period of employment hereunder,
          ---------------------                                             
except for periods of illness, disability, reasonable vacation periods, and
reasonable leaves of absence, Employee shall devote substantially all of her
business time, attention, skill, and efforts to the faithful performance of her
duties hereunder.  Employee shall be entitled to reasonable participation as a
member in community, civic, or similar organizations and the pursuit of personal
investments which do not present any material conflict of interest with
Employers, or otherwise unfavorably affect the performance of Employee's duties
pursuant to this Agreement.  The Employee shall have no employment contract or
other written or oral agreement concerning employment with any entity or person
other than Employers during the term of her employment under this Agreement.

     4.   Payments to Employee Upon Termination of Employment.  The Boards of
          ---------------------------------------------------                
Directors of Employers may terminate Employee's employment under this Agreement
at any time.  Employee

                                      -3-
<PAGE>
 
may voluntarily terminate her employment under this Agreement.  The rights and
obligations of Employers and Employee in the event of such termination are set
forth in this paragraph 4 as follows:

          (a) Termination Without Change of Control.  Employee shall have no
              -------------------------------------                         
     right to compensation or other benefits for any period after the
     termination of her employment, other than a termination after a Change of
     Control, regardless of whether the termination is with or without cause or
     the termination is voluntary or involuntary.
 
         (b)  Event of Termination in Connection With a Change of Control.  If,
              -----------------------------------------------------------      
     within six (6) months immediately following a Change of Control or within
                 -                                                            
     three (3) months immediately prior to such Change of Control, Employee's
            -                                                                
     employment with Employers is terminated by Employers or the status,
     character, capacity, and circumstances of Employee's employment have been
     materially altered by the Employers whether by a reduction in salary,
     responsibilities, authority, or benefits, then the Employers shall pay to
     Employee, or in the event of her subsequent death, her designated
     beneficiary or beneficiaries, or her estate, as the case may be, shall
     receive, as liquidated damages, in lieu of all other claims, a severance
     payment equal to one half ( 1/2) of Employee's then current annual salary
     plus one half ( 1/2) of any incentive compensation paid to Employee during
     the immediately preceding twelve (12) months, to be paid in full on the
                                       --                                   
     last day of the month following the date of said termination.  The
     responsibility for making such severance payment shall be allocated among
     Employers in whatever manner they deem appropriate.  In no event shall the
     total compensation paid to Employee upon the termination of her employment
     in connection with a Change of Control exceed the amount permitted by
     Section 280G of the Internal Revenue Code (as amended) or three times
     Employee's average annual compensation.

     5.   Regulatory Suspension.
          --------------------- 

                                      -4-
<PAGE>
 
          (a) If Employee is suspended and/or temporarily prohibited from
     participating in the conduct of the affairs of Employers by a notice served
     under Sections 8(e)(3) or (g)(1) of the Federal Deposit Insurance Act, 12
     U.S.C. (S)1818(e)(3) or (g)(1), then the obligations of Employers under
     this Agreement shall be suspended as of the date of service of such notice,
     unless stayed by appropriate proceedings.  If the charges in the notice are
     dismissed, then the Employers may, in their discretion, (i) pay Employee
     all or part of the compensation withheld while its contract obligations
     were suspended and (ii) reinstate in whole or in part any of its
     obligations which were suspended.

          (b) If  Employee is removed and/or permanently prohibited from
     participating in the conduct of the affairs of Employer by an order issued
     under Section 8(e)(4) or (g)(1) of the Federal Deposit Insurance Act, 12
     U.S.C. (S) 1818(e)(4) or (g)(1), then all obligations of the Employers
     under this Agreement shall terminate as of the effective date of the order.

     6.   Source of Payments.  All payments provided herein shall be paid in
          ------------------                                                
cash from the general funds of Employers as provided herein; no special or
separate fund shall be established by Employers; and no other segregation of
assets shall be made to assure payment.  Employee shall have no right, title, or
interest in or to any investments which Employers may make to meet the
obligations hereunder.

     7.   Nondisclosure of Confidential Information.  Employee acknowledges that
          -----------------------------------------                             
she possesses confidential information of a special and unique nature and value
affecting and relating to both Employers' and the Holding Company's business,
including, without limitation, customer lists, deposits, business records, other
trade secrets, and other similar confidential information relating to Employers
and/or the Holding Company and the business of each (all the foregoing being
hereinafter collectively referred to as "Confidential Information").  Employee
recognizes and acknowledges that all Confidential Information is the exclusive
property of Employers and/or the Holding Company, respectively, constitutes
trade secrets of Employers and/or the Holding

                                      -5-
<PAGE>
 
Company, is material and confidential, and greatly affects the goodwill and the
effective and successful conduct of the business of Employers and/or the Holding
Company.  As a material inducement to Employers to enter into this Agreement and
to employ Employee, Employee covenants and agrees that she will not at any time
during the term of her employment under this Agreement, and for a period of one
(1) year from the end of such employment, directly or indirectly, divulge,
 -                                                                        
reveal, or communicate any Confidential Information to any person, firm,
corporation, or entity whatsoever, or use any Confidential Information for her
own benefit or for the benefit of others.  Employee further acknowledges that
said Confidential Information has material commercial value to Employers and/or
the Holding Company so long as it is not known by competitors of Employers
and/or the Holding Company and that both the Employers and the Holding Company
have taken reasonable steps to keep all such information and trade secrets
confidential.

     8.   Injunctions.  In view of the irreparable harm and damage which
          -----------                                                   
Employers and/or the Holding Company would sustain as a result of a breach by
Employee of the covenants or agreements under paragraph 7 hereof, and in view of
the lack of an adequate remedy at law to protect the interests of Employers
and/or the Holding Company, Employers and/or the Holding Company shall have the
right to receive, and Employee hereby consents to the issuance of, a permanent
injunction enjoining Employee from any violation of the covenants and agreements
set forth in paragraph 7 hereof.  The foregoing remedy shall be in addition to,
and not in limitation of, any other rights or remedies to which Employers and/or
the Holding Company is/are or may be entitled at law or in equity respecting
this Agreement.  It is expressly agreed by the parties hereto that the Holding
Company is an intended third party beneficiary of paragraphs 7, 8, and 9 of this
Agreement and may enforce same against Employee as if it were a party hereto.

     9.   Attorneys' Fees.  In the event any party hereto is required to engage
          ---------------                                                      
in legal action against any other party hereto, either as plaintiff or
defendant, in order to enforce or defend any of its or her rights under this
Agreement, and such action results in a final judgment in favor of one or

                                      -6-
<PAGE>
 
more parties, then the party or parties against whom said final judgment is
obtained shall reimburse the prevailing party or parties for all legal fees and
expenses incurred by the prevailing party or parties in asserting or defending
its or her rights hereunder.

     10.  Effect of Prior Agreements.  This Agreement, that certain Indexed
          --------------------------                                       
Executive Salary Continuation Agreement dated as of September 9, 1994, as
amended and restated, and that certain Flexible Premium Life Insurance
Endorsement Method Split Dollar Plan Agreement, dated as of September 9, 1994,
as amended, contain the entire understanding between the parties hereto and
supersede any prior agreements relating to employment and any contemporaneous
oral agreement or understanding by, between, or among the Employers and
Employee.

     11.  General Provisions.
          ------------------ 

          (a) Nonassignability.  Neither this Agreement nor any right or
              ----------------                                          
     interest hereunder shall be assignable by Employee, her beneficiaries or
     legal representatives, without the written consent of Employers; provided,
     however, that nothing in this paragraph 11(a) shall preclude (i) Employee
     from designating a beneficiary to receive any benefits payable hereunder
     upon her death, or (ii) the executors, administrators, or other legal
     representatives of Employee or her estate from assigning any rights
     hereunder to the person or persons entitled thereto.

          (b) No Attachment.  Except as required by law, no right to receive
              -------------                                                 
     payments under this Agreement shall be subject to anticipation,
     commutation, alienation, sale, assignment, encumbrance, charge, pledge, or
     hypothecation, or to execution, attachment, levy, and any attempt,
     voluntary or involuntary, to effect any such action shall be null, void,
     and of no effect.

                                      -7-
<PAGE>
 
          (c) Binding Agreement.  This Agreement shall be binding upon, and
              -----------------                                            
     inure to the benefit of, the Employers and Employee and their respective
     heirs, successors, assigns, and legal representatives.

     12.  Modification and Waiver.
          ----------------------- 

          (a) Amendment of Agreement.  This Agreement may not be modified or
              ----------------------                                        
     amended except by an instrument in writing, signed by the parties hereto,
     and which specifically refers to this Agreement.

          (b) Waiver.  No term or condition of this Agreement shall be deemed to
              ------                                                            
     have been waived, nor shall there be any estoppel against the enforcement
     of any provision of this Agreement, except by written instrument of the
     party charged with such waiver or estoppel. No such written waiver shall be
     deemed a continuing waiver unless specifically stated therein, and each
     waiver shall operate only as to the specific term or condition waived and
     shall not constitute a waiver of such term or condition for the future or
     as to any act other than that specifically waived.

     13.  Severability.  If for any reason any provision of this Agreement is
          ------------                                                       
held invalid, such invalidity shall not affect any other provision of this
Agreement not held invalid, and each such other provision shall to the full
extent consistent with law continue in full force and effect.  If any provision
of this Agreement shall be held invalid in part, such invalidity shall in no way
affect the rest of such provision not held so invalid, and the rest of such
provision, together with all other provisions of this Agreement, shall to the
full extent consistent with law continue in full force and effect.

     14.  Headings.  The headings of paragraphs herein are included solely for
          --------                                                            
convenience of reference and shall not control the meaning or interpretation of
any of the provisions of this Agreement.

                                      -8-
<PAGE>
 
     15.  Governing Law.  This Agreement has been executed and delivered in the
          -------------                                                        
State of Georgia, and its validity, interpretation, performance, and enforcement
shall be governed by the laws of said State.

     16.  Rights of Third Parties.  Nothing herein expressed or implied is
          -----------------------                                         
intended to or shall be construed to confer upon or give to any person, firm, or
other entity, other than the parties hereto and their permitted assigns, any
rights or remedies under or by reason of this Agreement, except as expressly set
forth in paragraph 8 hereof.

     17.  Notices.  All notices, requests, demands, and other communications
          -------                                                           
provided for by this Agreement shall be in writing and shall be sufficiently
given if and when mailed in the United States by registered or certified mail,
or personally delivered, to the party entitled thereto at the address stated
below or to such changed address as the addressee may have given by a similar
notice:

          To Employers:  Chairman, Compensation Committee
                         Board of Directors
                         Premier Bancshares, Inc.
                         2180 Atlanta Plaza
                         950 E. Paces Ferry Road
                         Atlanta, Georgia  30326


          Copy to:       Steven S. Dunlevie, Esq.
                         Womble Carlyle Sandridge & Rice, PLLC
                         Suite 700
                         1275 Peachtree Street
                         Atlanta, Georgia 30309


          To Employee:   Ms. Linda S. George
                         P.O. Box 548
                         Snellville, Georgia 30078

                                      -9-
<PAGE>
 
          IN WITNESS WHEREOF, Employers have caused this Agreement to be
executed and its seal to be affixed hereunto by their duly authorized officers,
and Employee has signed this Agreement under seal, as of the Effective Date.

ATTEST:                              PREMIER BANK

/s/                                  By:  /s/ 
--------------------------                --------------------------
Secretary                                   Chairman of the Board 

(BANK SEAL)                                                       
                                                                  


ATTEST:                              THE BANK OF GWINNETT COUNTY


/s/                                  By:  /s/ Glenn S. White
--------------------------                --------------------------
Secretary                                   Glenn S. White                   
(BANK SEAL)                                 President      
                                                      




/s/                                   /s/ Linda S. George            (SEAL)
--------------------------           --------------------------------
Witness                               Linda S. George

                                      -10-