<PAGE>   1



                                                                     EXHIBIT 4.4


                     ====================================


                    COMMON SECURITIES GUARANTEE AGREEMENT


                        First Keystone Financial, Inc.


                         Dated as of August 26, 1997


                     ====================================
<PAGE>   2
                               TABLE OF CONTENTS

<TABLE>
<CAPTION>
                                                                                                                        Page
                                                                                                                        ----
<S>              <C>                                                                                                       <C>
                                                     ARTICLE I
                                           DEFINITIONS AND INTERPRETATION

SECTION 1.1.     Definitions and Interpretation  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   2
                 ------------------------------                                                                             

                                                     ARTICLE II
                                                     GUARANTEE

SECTION 2.1.     Guarantee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   3
                 ---------                                                                                                  
SECTION 2.2.     Waiver of Notice and Demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   3
                 ---------------------------                                                                                
SECTION 2.3.     Obligations Not Affected  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   3
                 ------------------------                                                                                   
SECTION 2.4.     Rights of Holders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   4
                 -----------------                                                                                          
SECTION 2.5.     Guarantee of Payment  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   4
                 --------------------                                                                                       
SECTION 2.6.     Subrogation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   5
                 -----------                                                                                                
SECTION 2.7.     Independent Obligations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   5
                 -----------------------                                                                                    

                                                    ARTICLE III
                                     LIMITATION OF TRANSACTIONS; SUBORDINATION

SECTION 3.1.     Limitation of Transactions  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   5
                 --------------------------                                                                                 
SECTION 3.2.     Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   6
                 -------                                                                                                    

                                                     ARTICLE IV
                                                    TERMINATION

SECTION 4.1.     Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   6
                 -----------                                                                                                

                                                     ARTICLE V
                                                   MISCELLANEOUS

SECTION 5.1.     Successors and Assigns  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   7
                 ----------------------                                                                                     
SECTION 5.2.     Amendments  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   7
                 ----------                                                                                                 
SECTION 5.3.     Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   7
                 -------                                                                                                    
SECTION 5.4.     Benefit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   8
                 -------                                                                                                    
SECTION 5.5.     Governing Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   8
                 -------------                                                                                              
</TABLE>
<PAGE>   3
                     COMMON SECURITIES GUARANTEE AGREEMENT


                 This GUARANTEE AGREEMENT (the "Common Securities Guarantee"),
dated as of August 26, 1997, is executed and delivered by First Keystone
Financial, Inc., a Pennsylvania corporation (the "Guarantor"), for the benefit
of the Holders (as defined herein) from time to time of the Common Securities
(as defined herein) of First Keystone Capital Trust I, a Delaware business
trust (the "Issuer").

                 WHEREAS, pursuant to an Amended and Restated Declaration of
Trust (the "Declaration"), dated as of August 26, 1997, among the Trustees of
the Issuer named therein, the Guarantor, as sponsor, and the holders from time
to time of undivided beneficial interests in the assets of the Issuer, the
Issuer is issuing on the date hereof 502 common securities designated the 9.70%
Common Securities (the "Common Securities"), having an aggregate stated
liquidation amount of $502,000;

                 WHEREAS, as incentive for the Holders to purchase the Common
Securities, the Guarantor desires to irrevocably and unconditionally agree, to
the extent set forth in this Common Securities Guarantee, to pay to the Holders
the Guarantee Payments (as defined herein) and to make certain other payments
on the terms and conditions set forth herein; and

                 WHEREAS, the Guarantor is also executing and delivering a
guarantee agreement (the "Series A Capital Securities Guarantee") for the
benefit of the holders of the Series A Capital Securities (as defined in the
Declaration) and upon consummation of the Exchange Offer (as defined in the
Declaration) will execute and deliver a guarantee agreement (the "Series B
Capital Securities Guarantee") for the benefit of the holders of the Series B
Capital Securities (as defined in the Declaration), each in substantially
identical terms to this Common Securities Guarantee, except that if an Event of
Default (as defined in the Declaration) has occurred and is continuing, the
rights of Holders to receive Guarantee Payments under this Common Securities
Guarantee are subordinated to the rights of holders of Capital Securities to
receive Guarantee Payments under the Series A Capital Securities Guarantee and
the Series B Capital Securities Guarantee, as the case may be;

                 NOW, THEREFORE, in consideration of the purchase by each
Holder, which purchase the Guarantor hereby acknowledges shall benefit the
Guarantor, the Guarantor executes and delivers this Common Securities Guarantee
for the benefit of the Holders.
<PAGE>   4
                                   ARTICLE I
                         DEFINITIONS AND INTERPRETATION

SECTION 1.1.     Definitions and Interpretation

                 In this Common Securities Guarantee, unless the context
otherwise requires:

                 (a)      Capitalized terms used in this Common Securities
         Guarantee but not defined in the preamble above have the respective
         meanings assigned to them in this Section 1.1;

                 (b)      Terms defined in the Declaration as at the date of
         execution of this Common Securities Guarantee have the same meaning
         when used in this Common Securities Guarantee unless otherwise defined
         in this Common Securities Guarantee;

                 (c)      a term defined anywhere in this Common Securities
         Guarantee has the same meaning throughout;

                 (d)      all references to "the Common Securities Guarantee"
         or "this Common Securities Guarantee" are to this Common Securities
         Guarantee as modified, supplemented or amended from time to time;

                 (e)      all references in this Common Securities Guarantee to
         Articles and Sections are to Articles and Sections of this Common
         Securities Guarantee unless otherwise specified; and

                 (f)      a reference to the singular includes the plural and
         vice versa.

                 "Guarantee Payments" means the following payments or
distributions, without duplication, with respect to the Common Securities, to
the extent not paid or made by the Issuer:  (i) any accrued and unpaid
Distributions that are required to be paid on such Common Securities to the
extent the Issuer has funds on hand legally available therefor at such time,
(ii) the redemption price, including all accrued and unpaid Distributions to
the date of redemption (the "Redemption Price") to the extent the Issuer has
funds on hand legally available therefor at such time, with respect to any
Common Securities called for redemption by the Issuer and (iii) upon a
voluntary or involuntary termination and liquidation of the Issuer (other than
in connection with the distribution of Debentures to the Holders in exchange
for Common Securities as provided in the Declaration), the lesser of (a) the
aggregate of the liquidation amount and all accumulated and unpaid
Distributions on the Common Securities to the date of payment, to the extent
the Issuer has funds on hand legally available therefor, and (b) the amount of
assets of the Issuer remaining available for distribution to Holders in
liquidation of the Issuer (in either case, the "Liquidation Distribution").  If
an Event of Default has occurred and is continuing, no Guarantee Payments with
respect to the Common Securities shall be made until holders of Capital





                                       2
<PAGE>   5
Securities shall be paid in full the Guarantee Payments to which they are
entitled under the Series A Capital Securities Guarantee and the Series B
Capital Securities Guarantee.

                 "Holder" means any holder, as registered on the books and
records of the Issuer, of any Common Securities.

                 "Other Guarantees" means all guarantees to be issued by the
Guarantor with respect to common securities (if any) similar to the Common
Securities issued by other trusts to be established by the Guarantor (if any),
in each case similar to the Issuer.


                                   ARTICLE II
                                   GUARANTEE

SECTION 2.1.     Guarantee

                 The Guarantor irrevocably and unconditionally agrees to pay in
full to the Holders the Guarantee Payments (without duplication of amounts
theretofore paid by the Issuer), as and when due, regardless of any defense,
right of set-off or counterclaim which the Issuer may have or assert.  The
Guarantor's obligation to make a Guarantee Payment may be satisfied by direct
payment of the required amounts by the Guarantor to the Holders or by causing
the Issuer to pay such amounts to the Holders.

SECTION 2.2.     Waiver of Notice and Demand

                 The Guarantor hereby waives notice of acceptance of this
Common Securities Guarantee and of any liability to which it applies or may
apply, presentment, demand for payment, any right to require a proceeding first
against the Issuer or any other Person before proceeding against the Guarantor,
protest, notice of nonpayment, notice of dishonor, notice of redemption and all
other notices and demands.

SECTION 2.3.     Obligations Not Affected

                 The obligations, covenants, agreements and duties of the
Guarantor under this Common Securities Guarantee shall in no way be affected or
impaired by reason of the happening from time to time of any of the following:

                 (a)  the release or waiver, by operation of law or otherwise,
         of the performance or observance by the Issuer of any express or
         implied agreement, covenant, term or condition relating to the Common
         Securities to be performed or observed by the Issuer;

                 (b)  the extension of time for the payment by the Issuer of
         all or any portion of the Distributions, Redemption Price, Liquidation
         Distribution or any other sums





                                       3
<PAGE>   6
         payable under the terms of the Common Securities or the extension of
         time for the performance of any other obligation under, arising out
         of, or in connection with, the Common Securities (other than an
         extension of time for payment of Distributions, Redemption Price,
         Liquidation Distribution or other sum payable that results from the
         extension of any interest payment period on the Debentures permitted
         by the Indenture);

                 (c)      any failure, omission, delay or lack of diligence on
         the part of the Holders to enforce, assert or exercise any right,
         privilege, power or remedy conferred on the Holders pursuant to the
         terms of the Common Securities, or any action on the part of the
         Issuer granting indulgence or extension of any kind;

                 (d)      the voluntary or involuntary liquidation,
         dissolution, sale of any collateral, receivership, insolvency,
         bankruptcy, assignment for the benefit of creditors, reorganization,
         arrangement, composition or readjustment of debt of, or other similar
         proceedings affecting, the Issuer or any of the assets of the Issuer;

                 (e)      any invalidity of, or defect or deficiency in, the
         Common Securities;

                 (f)      the settlement or compromise of any obligation
         guaranteed hereby or hereby incurred; or

                 (g)      any other circumstance whatsoever that might
         otherwise constitute a legal or equitable discharge or defense of a
         guarantor, it being the intent of this Section 2.3 that the
         obligations of the Guarantor with respect to the Guarantee Payments
         shall be absolute and unconditional under any and all circumstances.

There shall be no obligation of the Holders to give notice to, or obtain
consent of, the Guarantor with respect to the happening of any of the
foregoing.

SECTION 2.4.     Rights of Holders

                 The Guarantor expressly acknowledges that any Holder may
institute a legal proceeding directly against the Guarantor to enforce its
rights under this Common Securities Guarantee, without first instituting a
legal proceeding against the Issuer or any other Person.

SECTION 2.5.     Guarantee of Payment

                 This Common Securities Guarantee creates a guarantee of payment
and not of collection.





                                       4
<PAGE>   7
SECTION 2.6.     Subrogation

                 The Guarantor shall be subrogated to all (if any) rights of
the Holders against the Issuer in respect of any amounts paid to such Holders
by the Guarantor under this Common    Securities Guarantee; provided, however,
that the Guarantor shall not (except to the extent required by mandatory
provisions of law) be entitled to enforce or exercise any rights which it may
acquire by way of subrogation or any indemnity, reimbursement or other
agreement, in all cases as a result of payment under this Common Securities
Guarantee, if, at the time of any such payment, any amounts are due and unpaid
under this Common Securities Guarantee.  If any amount shall be paid to the
Guarantor in violation of the preceding sentence, the Guarantor agrees to hold
such amount in trust for the Holders and to pay over such amount to the
Holders.

SECTION 2.7.     Independent Obligations

                 The Guarantor acknowledges that its obligations hereunder are
independent of the obligations of the Issuer with respect to the Common
Securities and that the Guarantor shall be liable as principal and as debtor
hereunder to make Guarantee Payments pursuant to the terms of this Common
Securities Guarantee notwithstanding the occurrence of any event referred to in
subsections (a) through (g), inclusive, of Section 2.3 hereof.


                                  ARTICLE III
                   LIMITATION OF TRANSACTIONS; SUBORDINATION

SECTION 3.1.     Limitation of Transactions

                 So long as any Common Securities remain outstanding, the
Guarantor will not (i) declare or pay any dividends or distributions on, or
redeem, purchase, acquire or make a liquidation payment with respect to, any of
the Guarantor's capital stock (which includes common stock and preferred
stock), (ii) make any payment of principal of, or premium, if any, or interest
on or repay, repurchase or redeem any debt securities of the Guarantor
(including Other Debentures) that rank pari passu with or junior in right of
payment to the Debentures or (iii) make any guarantee payments with respect to
any guarantee by the Guarantor of the debt securities of any subsidiary of the
Guarantor (including under Other Guarantees) if such guarantee ranks pari passu
with or junior in right of payment to the Debentures (other than (a) dividends
or distributions in shares of, or options, warrants or rights to subscribe for
or purchase shares of, common stock of the Guarantor, (b) any declaration of a
dividend in connection with the implementation of a stockholders' rights plan,
or the issuance of stock under any such plan in the future, or the redemption
or repurchase of any such rights pursuant thereto, (c) payments under the
Series A Capital Securities Guarantee and the Series B Capital Securities
Guarantee, (d) as a result of a reclassification of the Guarantor's capital
stock or the exchange or conversion of one class or series of the





                                       5
<PAGE>   8
Guarantor's capital stock for another class or series of the Guarantor's
capital stock, (e) the purchase of fractional interests in shares of the
Guarantor's capital stock pursuant to the conversion or exchange provisions of
such capital stock or the security being converted or exchanged and (f)
purchases of common stock related to the issuance of common stock or rights
under any of the Guarantor's benefit plans for its directors, officers or
employees or any of the Guarantor's dividend reinvestment plans) if at such
time (i) there shall have occurred any event of which the Guarantor has actual
knowledge that (a) is, or with the giving of notice or the lapse of time, or
both, would be, an Event of Default and (b) in respect of which the Guarantor
shall not have taken reasonable steps to cure, (ii) if such Debentures are held
by the Property Trustee, the Guarantor shall be in default with respect to its
payment of any obligations under the Capital Securities Guarantee or (iii) the
Guarantor shall have given notice of its election of the exercise of its right
to extend the interest payment period pursuant to Section 16.01 of the
Indenture and any such extension shall be continuing.

SECTION 3.2.     Ranking

                 This Common Securities Guarantee will constitute an unsecured
obligation of the Guarantor and will rank (i) subordinate and junior in right
of payment to Senior Indebtedness (as defined in the Indenture), to the same
extent and in the same manner that the Debentures are subordinated to Senior
Indebtedness pursuant to the Indenture, (ii) pari passu with the Debentures,
the Other Debentures and with any Other Guarantee, and (iii) senior to the
Guarantor's capital stock.


                                   ARTICLE IV
                                  TERMINATION

SECTION 4.1.     Termination

                 This Common Securities Guarantee shall terminate (i) upon full
payment of the Redemption Price of all Common Securities, (ii) upon the
distribution of all of the Debentures to all the Holders and the holders of the
Capital Securities or (iii) upon full payment of the amounts payable in
accordance with the Declaration upon liquidation of the Issuer.
Notwithstanding the foregoing, this Common Securities Guarantee will continue
to be effective or will be reinstated, as the case may be, if at any time any
Holder must restore payment of any sums paid under the Common Securities or
under this Common Securities Guarantee.





                                       6
<PAGE>   9
                                   ARTICLE V
                                 MISCELLANEOUS

SECTION 5.1.     Successors and Assigns

                 All guarantees and agreements contained in this Common
Securities Guarantee shall bind the successors, assigns, receivers, trustees
and representatives of the Guarantor and shall inure to the benefit of the
Holders then outstanding.

SECTION 5.2.     Amendments

                 Except with respect to any changes which do not adversely
affect the rights of Holders (in which case no consent of Holders will be
required), this Common Securities Guarantee may only be amended with the prior
approval of the Holders of at least a majority in liquidation amount of all the
outstanding Common Securities.  The provisions of Section 12.2 of the
Declaration with respect to meetings of holders of the Securities apply to the
giving of such approval.

SECTION 5.3.     Notices

                 All notices provided for in this Common Securities Guarantee
shall be in writing, duly signed by the party giving such notice, and shall be
delivered, telecopied or mailed by registered or certified mail, as follows:

                 (a)      if given to the Issuer, in care of the Administrative
         Trustee at the Issuer's mailing address set forth below (or such other
         address as the Issuer may give notice of to the Holders):

                                  First Keystone Capital Trust I
                                  c/o First Keystone Financial, Inc.
                                  22 West State Street
                                  Media, Pennsylvania 19063
                                  Attention:       Thomas M. Kelly
                                                   Administrative Trustee
                                  Telecopy:        (610) 892-5108

                 (b)      if given to the Guarantor, at the Guarantor's mailing
         address set forth below (or such other address as the Guarantor may
         give notice of to the Holders):





                                       7
<PAGE>   10
                                  First Keystone Financial, Inc.
                                  22 West State Street
                                  Media, Pennsylvania 19063
                                  Attention:       Thomas M. Kelly
                                                   Executive Vice President and
                                                   Chief Financial Officer
                                  Telecopy:        (610) 892-5108

                 (c)      if given to any Holder, at the address set forth on
the books and records of the Issuer.

                 All such notices shall be deemed to have been given when
received in person, telecopied with receipt confirmed, or mailed by first class
mail, postage prepaid except that if a notice or other document is refused
delivery or cannot be delivered because of a changed address of which no notice
was given, such notice or other document shall be deemed to have been delivered
on the date of such refusal or inability to deliver.

SECTION 5.4.     Benefit

                 This Common Securities Guarantee is solely for the benefit of
the Holders and is not separately transferable from the Common Securities.

SECTION 5.5.     Governing Law

                 THIS COMMON SECURITIES GUARANTEE SHALL BE GOVERNED BY, AND
CONSTRUED AND INTERPRETED IN ACCORDANCE WITH, THE LAWS OF THE STATE OF NEW
YORK, WITHOUT REGARD TO CONFLICTS OF LAWS PRINCIPLES THEREOF.





                                       8
<PAGE>   11
                  THIS COMMON SECURITIES GUARANTEE is executed as of the day and
year first above written.

                    FIRST KEYSTONE FINANCIAL, INC.
                    
                    
                    
                    By:                                                   
                         -------------------------------------------------
                         Name:  Thomas M. Kelly
                         Title: Executive Vice President and Chief
                                Financial Officer





                                       9