<Page>

                                                                    EXHIBIT 99.4

                    CERTIFICATION OF CHIEF FINANCIAL OFFICER
                             AS ADOPTED PURSUANT TO
                SECTION 302(a) OF THE SARBANES-OXLEY ACT OF 2002

I, Kathleen S. Dvorak, certify that:

     1.   I have reviewed this annual report on Form 10-K of United Stationers
          Inc. and United Stationers Supply Co. (together, the "registrant");

     2.   Based on my knowledge, this annual report does not contain any untrue
          statement of a material fact or omit to state a material fact
          necessary to make the statements made, in light of the circumstances
          under which such statements were made, not misleading with respect to
          the period covered by this annual report;

     3.   Based on my knowledge, the financial statements, and other financial
          information included in this annual report, fairly present in all
          material respects the financial condition, results of operations and
          cash flows of the registrant as of, and for, the periods presented in
          this annual report;

     4.   The registrant's other certifying officers and I are responsible for
          establishing and maintaining disclosure controls and procedures (as
          defined in Exchange Act Rules 13a-14 and 15d-14) for the registrant
          and we have:

          a)   designed such disclosure controls and procedures to ensure that
               material information relating to the registrant, including its
               consolidated subsidiaries, is made known to us by others within
               those entities, particularly during the period in which this
               annual report is being prepared;

          b)   evaluated the effectiveness of the registrant's disclosure
               controls and procedures as of a date within 90 days prior to the
               filing date of this annual report (the "Evaluation Date"); and

          c)   presented in this annual report our conclusions about the
               effectiveness of the disclosure controls and procedures based on
               our evaluation as of the Evaluation Date;

     5.   The registrant's other certifying officers and I have disclosed, based
          on our most recent evaluation, to the registrant's auditors and the
          audit committee of registrant's Board of Directors (or persons
          performing the equivalent function):

          a)   all significant deficiencies in the design or operation of
               internal controls which could adversely affect the registrant's
               ability to record, process, summarize and report financial data
               and have identified for the registrant's auditors any material
               weaknesses in internal controls; and

          b)   any fraud, whether or not material, that involves management or
               other employees who have a significant role in the registrant's
               internal controls; and

     6.   The registrant's other certifying officers and I have indicated in
          this annual report whether or not there were significant changes in
          internal controls or in other factors that could significantly affect
          internal controls subsequent to the date of our most recent
          evaluation, including any corrective actions with regard to
          significant deficiencies and material weaknesses.

Date: March 28, 2003           /s/ Kathleen S. Dvorak
                               ---------------------------------------

                               Kathleen S. Dvorak
                               Senior Vice President and Chief Financial Officer