<PAGE>

                                                                    EXHIBIT 31.2

                    CERTIFICATION OF CHIEF FINANCIAL OFFICER

   PURSUANT TO RULES 13a-14 AND 15d-14 OF THE SECURITIES EXCHANGE ACT OF 1934
                AND SECTION 302 OF THE SARBANES-OXLEY ACT OF 2002

I, Thomas M. Kelly, certify that:

1.  I have reviewed this quarterly report on Form 10-K of First Keystone
    Financial, Inc. (the "registrant");

2.  Based on my knowledge, this report does not contain any untrue
    statement of a material fact or omit to state a material fact necessary
    to make the statements made, in light of the circumstances under which
    such statements were made, not misleading with respect to the period
    covered by this report;

3.  Based on my knowledge, the financial statements, and other financial
    information included in this report, fairly present in all material
    respects the financial condition, results of operations and cash flows
    of the registrant as of, and for, the periods presented in this report;

4.  The registrant's other certifying officers and I are responsible for
    establishing and maintaining disclosure controls and procedures (as
    defined in Exchange Act Rules 13a-15(e) and 15d-15(e)) for the
    registrant and have:

    (a)      Designed such disclosure controls and procedures, or caused
             such disclosure controls and procedures to be designed under
             our supervision, to ensure that material information relating
             to the registrant, including its consolidated subsidiaries, is
             made known to us by others within those entities, particularly
             during the period in which this report is being prepared;

    (b)      Evaluated the effectiveness of the registrant's disclosure
             controls and procedures and presented in this report our
             conclusions about the effectiveness of the disclosure controls
             and procedures, as of the end of the period covered by this
             report based on such evaluation; and

    (c)      Disclosed in this report any change in the registrant's
             internal control over financial reporting that occurred during
             the registrant's most recent fiscal quarter (the registrant's
             fourth fiscal quarter in the case of an annual report) that
             has materially affected, or is reasonably likely to materially
             affect, the registrant's internal control over financial
             reporting; and

5.  The registrant's other certifying officers and I have disclosed, based
    on our most recent evaluation of internal control over financial
    reporting, to the registrant's auditors and the audit committee of
    registrant's board of directors (or persons performing the equivalent
    functions):

    (a)      All significant deficiencies and material weaknesses in the
             design or operation of internal control over financial
             reporting which are reasonably likely to adversely affect the
             registrant's ability to record, process, summarize and report
             financial information; and

    (b)      Any fraud, whether or not material, that involves management
             or other employees who have a significant role in the
             registrant's internal control over financial reporting.

Date: December 23, 2003                   /s/ Thomas M. Kelly
                                          --------------------------------------
                                          Thomas M. Kelly
                                          President and Chief Financial Officer

                                                                              89