<PAGE>   1


                                  UNITED STATES
                       SECURITIES AND EXCHANGE COMMISSION
                             WASHINGTON, D.C. 20549

                                    FORM 10-Q


[X]      Quarterly Report Pursuant to Section 13 or 15(d) of the Securities
         Exchange Act of 1934

         FOR THE QUARTER ENDED MAY 3, 1997

                                       OR

[ ]      Transition Report Pursuant to Section 13 or 15(d) of the Securities
         Exchange Act of 1934

For the transition period from                       to
                                -------------------      -------------------

                      ------------------------------------
                          COMMISSION FILE NUMBER 1-8578
                      ------------------------------------

                              MCRAE INDUSTRIES, INC
             (Exact name of registrant as specified in its charter)

                    DELAWARE                                 56-0706710
         (State or other jurisdiction of                   (I.R.S. Employer
       incorporation or organization)                    Identification No.)

                              400 NORTH MAIN STREET
                        MT. GILEAD, NORTH CAROLINA 27306
                    (Address of principal executive offices)

                         TELEPHONE NUMBER (910)439-6147
              (Registrant's telephone number, including area code)


Indicate by check mark whether the registrant (1) has filed all reports required
to be filed by Section 13 or 15 (d) of the Securities Exchange Act of 1934
during the preceding 12 months (or for such shorter periods that the registrant
was required to file such reports), and (2) has been subject to such filing
requirements for the past 90 days.

                 Yes  X                                   No
                     ---                                     ---

Indicate the number of shares outstanding of each of the issuer's classes of
common stock, as of the latest practical date.

Common Stock, $1 Par Value--Class A  1,816,737 shares as of June 9, 1997
Common Stock, $1 Par Value--Class B    951,762 shares as of June 9, 1997


                                        1

<PAGE>   2



                     MCRAE INDUSTRIES, INC. AND SUBSIDIARIES

                                      INDEX


                                                                        Page No.
                                                                        --------


                          PART 1. FINANCIAL INFORMATION


ITEM 1.    CONDENSED CONSOLIDATED FINANCIAL STATEMENTS

           Condensed Consolidated Balance Sheet                           3-4

           Condensed Consolidated Statement of Operations                  5

           Condensed Consolidated Statement of Cash Flows                  6

           Notes to Condensed Consolidated Financial Statements            7



ITEM 2.    MANAGEMENT'S DISCUSSION AND ANALYSIS OF FINANCIAL
            CONDITION AND RESULTS OF OPERATIONS                           8-9


ITEM 3.    QUANTITATIVE AND QUALITATIVE DISCLOSURES ABOUT
            MARKET RISK                                                    9


                           PART II. OTHER INFORMATION


ITEM 1.    LEGAL PROCEEDINGS                                              10

ITEM 2.    CHANGES IN SECURITIES                                          10

ITEM 3.    DEFAULT UPON SENIOR SECURITIES                                 10

ITEM 4.    SUBMISSION OF MATTERS TO A VOTE OF SECURITIES HOLDERS          10

ITEM 5.    OTHER INFORMATION                                              10

ITEM 6.    EXHIBITS AND REPORTS ON FORM 8-K                               10


           SIGNATURES                                                     10





                                        2

<PAGE>   3



PART I. FINANCIAL INFORMATION

ITEM 1. CONDENSED CONSOLIDATED FINANCIAL STATEMENTS


                     MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                      CONDENSED CONSOLIDATED BALANCE SHEET
                                     ASSETS
                 (In thousands, except share and per share data)


<TABLE>
<CAPTION>
                                                May 3, 1997     August 3, 1996
                                                (Unaudited)          (Note)
                                                -----------          ------


<S>                                                <C>             <C>    
ASSETS

Current assets:

 Cash and cash equivalents                         $ 1,915         $   581

 Securities                                             65              65

 Accounts and notes receivable, net                  8,356          10,606

 Inventories                                        13,179          12,640

 Net investment in capitalized leases                  963             966

 Prepaid expenses and other current assets             294             210
                                                   -------         -------

   Total current assets                             24,772          25,068
                                                   -------         -------


Property, plant and equipment, net                   7,002           7,172


Other assets:

 Receivables, related entities                       2,561           2,359

 Net investment in capitalized leases                1,660           1,798

 Notes receivable                                    1,586             952

 Real estate held for investment                       486             478

 Goodwill                                              639             674

 Other                                               1,117           1,060
                                                   -------         -------

   Total other assets                                8,049           7,321
                                                   -------         -------

                                                   $39,823         $39,561
                                                   =======         =======
</TABLE>



                   See notes to quarterly financial statements

                                        3

<PAGE>   4



                     MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                      CONDENSED CONSOLIDATED BALANCE SHEET
                      LIABILITIES AND SHAREHOLDERS' EQUITY
                 (In thousands, except share and per share data)


<TABLE>
<CAPTION>
                                                   May 3, 1997    August 3, 1996
                                                   (Unaudited)        (Note)
                                                   -----------        ------

<S>                                                    <C>             <C>    
LIABILITIES AND SHAREHOLDERS' EQUITY

Current liabilities:

 Notes payable to bank                                 $   699         $   789

 Accounts payable                                        2,505           2,897

 Accrued employee benefits                                 887             846

 Deferred revenues                                       1,313           1,454

 Accrued payroll and payroll taxes                         667             790

 Income taxes                                              337             759

 Other                                                     470             580
                                                       -------         -------

   Total current liabilities                             6,878           8,115
                                                       -------         -------


Notes payable to banks, net of current portion           5,946           6,285

Minority Interest                                          859             797


Shareholders' Equity:
 Common stock:
  Class "A", $1 par; Authorized 5,000,000
    shares; Issued and outstanding, 1,809,964
    and 1,788,286, shares, respectively                  1,810           1,788
  Class "B", $1 par, Authorized 2,500,000
    shares; Issued and outstanding 958,535
    and 951,213 shares, respectively                       959             951

 Additional Paid-in Capital                                734             705

 Retained Earnings                                      22,637          20,920
                                                       -------         -------

    Total Shareholders' Equity                          26,140          24,364
                                                       -------         -------

                                                       $39,823         $39,561
                                                       =======         =======
</TABLE>


NOTE - The condensed consolidated balance sheet at August 3, 1996 has been
derived from the audited financial statements at that date but does not include
all of the information and footnotes required by generally accepted accounting
principles for complete financial statements.



                   See notes to quarterly financial statements

                                        4

<PAGE>   5



                     MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                 CONDENSED CONSOLIDATED STATEMENT OF OPERATIONS
                 (In thousands, except share and per share data)
                                   (Unaudited)


<TABLE>
<CAPTION>
                                          Three Months Ended                  Nine Months Ended
                                        May 3,          April 27,           May 3,          April 27,
                                         1997              1996              1997              1996
                                     -----------       -----------       -----------       -----------

<S>                                  <C>               <C>               <C>               <C>        
Net revenues                         $    12,646       $    11,069       $    44,934       $    31,623

 Costs and expenses:

  Cost of revenues                         9,362             7,830            32,623            22,108

  Selling and administrative               2,917             2,700             9,093             7,818

  Other expense (income), net               (162)              (89)              (55)             (289)
                                     -----------       -----------       -----------       -----------

Total costs and expenses                  12,117            10,441            41,661            29,637
                                     -----------       -----------       -----------       -----------

Earnings before income taxes
  and minority interest                      529               628             3,273             1,986

Provision for income taxes                   104               236             1,176               752

Minority shareholder's interest
  in earnings of subsidiary                   (2)                8                62                57
                                     -----------       -----------       -----------       -----------

Net earnings                         $       427       $       384       $     2,035       $     1,177
                                     ===========       ===========       ===========       ===========

Net earnings per Common Share        $       .15       $       .14       $       .74       $       .43
                                     -----------       -----------       -----------       -----------

Weighted average number of
  Common Shares Outstanding            2,768,499         2,728,185         2,759,683         2,729,491
                                     -----------       -----------       -----------       -----------
</TABLE>



                  See notes to quarterly financial statements.

                                        5

<PAGE>   6




                     MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                 CONDENSED CONSOLIDATED STATEMENT OF CASH FLOWS
                                 (In thousands)
                                   (Unaudited)


<TABLE>
<CAPTION>
                                                          Nine Months Ended
                                                     May 3, 1997  April 27, 1996
                                                     -----------  --------------


<S>                                                     <C>           <C>     
Net cash provided by (used in) operating
  activities                                            $ 3,490       $  (806)

Cash flows from investing activities:

  Proceeds from securities                                  -0-         2,025

  Capital expenditures                                     (294)         (526)

  Purchase of other assets                                 (142)         (340)

  Net advances of long term
   receivables                                             (867)          (50)

  Purchase of Minority Interests                            -0-           184
                                                        -------       -------

Net cash provided by (used in)
  investing activities                                   (1,303)          925
                                                        -------       -------

Cash flows from financing activities:

  Principal repayments of notes payable                    (429)          -0-

  Purchase Treasury Stock                                   -0-           (24)

  Proceeds from exercise of stock options                    58           -0-

  Dividends paid                                           (482)         (467)
                                                        -------       -------

Net cash used in financing activities                      (853)         (491)
                                                        -------       -------

Net increase (decrease) in cash and cash equivalents      1,334           372

Cash and Cash Equivalents at Beginning of Period            581           628
                                                        -------       -------

Cash and Cash Equivalents at End of Period              $ 1,915       $ 1,256
                                                        =======       =======
</TABLE>



                  See notes to quarterly financial statements.

                                        6

<PAGE>   7




                     MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
              NOTES TO CONDENSED CONSOLIDATED FINANCIAL STATEMENTS
                                   (UNAUDITED)


NOTE A - BASIS OF PRESENTATION

The accompanying unaudited consolidated financial statements have been prepared
in accordance with generally accepted accounting principles for interim
financial information and with the instructions to Form 10-Q and Article 10
Regulation S-X. Accordingly, they do not include all of the information and
footnotes required by generally accepted accounting principles for complete
financial statements. In the opinion of management, all adjustments (consisting
of normal recurring accruals) considered necessary for a fair presentation have
been included. Operating results for the nine months ended May 3, 1997 are not
necessarily indicative of the results that may be expected for the year ending
August 2, 1997. For further information, refer to the consolidated financial
statements and footnotes thereto included in the McRae Industries, Inc. annual
report on Form 10-K for the year ended August 3, 1996.

Certain reclassifications have been made to the prior year's financial
statements to conform with the current year's presentation.

NOTE B - INVENTORIES

An actual valuation of inventory under the LIFO method can be made only at the
end of each year based on the inventory levels and costs at that time.
Accordingly, interim LIFO calculations must necessarily be based on management's
estimates of expected year-end inventory levels and costs. Because these are
subject to many forces beyond management's control, interim calculations, if
any, are subject to the final year-end LIFO inventory valuation.

The components of inventory consist of the following (in thousands):


<TABLE>
<CAPTION>
                                         May 3, 1997     August 3, 1996
                                         -----------     --------------
<S>                                        <C>               <C>    
Raw materials                              $ 2,186           $ 2,288
Work in process                              1,073             1,142
Finished goods                               9,920             9,210
                                           -------------------------
                                           $13,179           $12,640
                                           =========================
</TABLE>

NOTE C - SUBSEQUENT EVENTS

On February 14, 1997, the Company announced plans to consolidate its western and
work boot production facilities by closing the manufacturing, retail and
administrative operations at the Dresden, Tennessee plant. The plant's
manufacturing and administrative operations were shut down as of June 2, 1997.
The Company intends to continue to use the facilities for warehousing and
distribution.

The Company evaluated the fair value of the Dresden, Tennessee location's assets
based on appraisals of the land, building and machinery. Based on the appraised
values the Company does not believe that the carrying value of the assets have
been impaired.

On May 28, 1997, the Company declared a cash dividend of 9.0 cents per share on
its Class A Common Stock payable on June 27, 1997 to shareholders of record on
June 13, 1997.


                                        7

<PAGE>   8



                     MCRAE INDUSTRIES, INC. AND SUBSIDIARIES

ITEM 2. MANAGEMENT'S DISCUSSION AND ANALYSIS OF FINANCIAL CONDITION AND
        RESULTS OF OPERATIONS

The following discussion should be read in conjunction with the attached
unaudited consolidated financial statements and notes thereto, and with the
Company's Annual Report on Form 10-K for the fiscal year ended August 3, 1996,
including the financial information and management's discussion and analysis
contained or incorporated by reference therein.

FINANCIAL CONDITION AND LIQUIDITY

For the quarter ended May 3, 1997, the Company continued to exhibit a strong
working capital position. The current ratio for the nine months ended May 3,
1997 was 3.6 to 1 as compared to 3.1 to 1 on August 3, 1996.

Cash, cash equivalents and marketable securities amounted to $1,980,000, an
increase of $1,334,000 over the amount reported at August 3, 1996. Cash
generated from operating activities totaled $3,490,000. Operating income,
adjusted for depreciation and amortization, contributed approximately $3.4
million. Accounts and notes receivable primarily associated with the footwear
and bar code segments provided an additional $2.2 million. Inventory primarily
attributable to the office products segment used $1.3 million. Accounts payable,
accrued payroll and related taxes and deferred revenues used approximately
$600,000 of cash. Estimated income tax payments were also responsible for a cash
outflow of $434,000.

Net cash used in investing activities amounted to approximately $1.3 million.
Capital expenditures primarily for manufacturing machinery and equipment, office
furniture and fixtures, computer equipment and automobiles amounted to $294,000.
Advances to related parties for working capital purposes used $223,000 of cash
while the financing and leasing segment used another $644,000. Officer life
insurance premiums and land purchased for investment used an additional $142,000
of cash.

Dividend payments of $482,000 and principal payments on long term debt of
$429,000 were the major factors in the net cash used in financing activities.
These outflows were partially offset by proceeds from the sale of stock
amounting to $58,000.

The Company maintains credit lines with several banks totalling $3.75 million.
As of May 3, 1997, the Company had utilized $400,000. It is management's opinion
that current cash and cash equivalents, available lines of credit and cash flows
from operations will be sufficient to provide for the Company's future working
capital and debt repayment requirements.

THIRD QUARTER FISCAL 1997 COMPARED TO THIRD QUARTER FISCAL 1996

Consolidated net revenues for the third fiscal quarter of 1997 amounted to
$12,646,000 as compared to $11,069,000 for the same period of fiscal 1996, an
increase of 14%. The footwear segment contributed approximately $1.6 million of
the net revenue increase for the quarter primarily as a result of the addition
of the western and work boot unit. The office products segment posted a net
revenue increase of almost $900,000 for the third fiscal quarter of 1997
compared to the same period of 1996. These increases were partially offset by
the bar code unit which recorded a net revenue reduction of approximately
$900,000 for the respective reporting periods.

In April 1997, the Company received a new contract from the U.S. Government to
supply military boots over the next year. The government may, at its discretion,
extend the contract for each of the next four years for additional option
quantities. While this is a positive development for the footwear segment,
management believes that the seasonal trends of reduced demand during the spring
and summer months associated with the western and work boot unit will partially
offset the anticipated revenue increase from the new government contract for the
next fiscal quarter.

Consolidated gross profit for the third quarter of fiscal 1997 posted a small
gain as compared to the same period of fiscal 1996. As a percent of net revenue,
gross profit

                                        8

<PAGE>   9



declined from 29.3% for the third quarter of 1996 to 26.0% for the third quarter
of 1997. This decrease was primarily the result of the product mix being more
heavily influenced by the lower margin footwear segment which accounted for 39%
of net revenues for the current fiscal quarter versus 30% for the same quarter
last year.

Selling, General and Administrative (SG&A) expenses increased by approximately
$200,000. As a percent of net revenues, however, SG&A expenses decreased from
24% for the third fiscal quarter of 1996 to 23% for the same period in fiscal
1997.


FIRST NINE MONTHS FISCAL 1997 COMPARED TO FIRST NINE MONTHS FISCAL 1996

Consolidated net revenues for the nine months ended May 3, 1997 were
$44,934,000, an increase of 42% over the $31,623,000 reported for the comparable
period of fiscal 1996. Each of the three major business segments contributed to
the $13,311,000 increase in net revenues. The footwear segment was responsible
for $11,761,000, or 88%, of the increase primarily as a result of the
acquisition of the western and work boot unit, which contributed $10,232,000 of
the increase for the fiscal reporting period. The office products and bar code
segments were responsible for 9% and 2% of the increase respectively by
contributing $1,172,000 and $327,000 of the increase, respectively.

Gross profit increased by $2.8 million as a result of greater net revenues
reported for the current fiscal period as compared to the same period of fiscal
1996. Gross profit as a percent of net revenues decreased from 30.1% for the
1996 fiscal period to 27.4% for the comparable period of fiscal 1997. This
decline in gross profit percentage was primarily attributable to the increased
concentration of lower margin sales from the footwear segment in the product mix
for the current reporting period as compared to the prior year period.

Selling, General and Administrative expenses approximated $9.1 million for the
current nine month fiscal period, an increase of 16% over $7.8 million reported
for the first nine months of fiscal 1996. The addition of the western and work
boot unit was responsible for approximately $1.1 million of the total $1.3
million increase. As a percent of net revenues, however, SG&A expenses decreased
to 20.2% for the current fiscal nine month period as compared to 24.7% for the
same period for fiscal 1996.

ITEM 3.  QUANTITATIVE AND QUALITATIVE DISCLOSURES ABOUT MARKET RISK

Not applicable.



                                        9

<PAGE>   10



PART II. OTHER INFORMATION


ITEM 1. LEGAL PROCEEDINGS

Reference is made to Item 3 of the Company's Annual Report to Shareholders on
Form 10-K for the fiscal year ended August 3, 1996.


ITEMS 2,3,4 AND 5

These items are inapplicable and have been omitted.


ITEM 6. EXHIBITS AND REPORTS ON FORM 8-K

(a) Exhibits

    Exhibit 27    Financial Data Schedule. (Filed in electronic format only.
                  Pursuant to Rule 402 of Regulations S-T, this schedule shall
                  not be deemed filed for purposes of Section 11 of the
                  Securities Act of 1933 or Section 18 of the Securities
                  Exchange Act of 1934.)


(b) No reports on Form 8-K were filed during the quarter ended May 3, 1997.




SIGNATURES

Pursuant to the requirements of the Securities Exchange Act of 1934, the
registrant has duly caused this report to be signed on its behalf by the
undersigned thereunto duly authorized.


                                         MCRAE INDUSTRIES, INC.
                                             (Registrant)



  DATE: June 17, 1997                 By: /s/ D. Gary McRae
       ------------------                 -----------------
                                      D. Gary McRae
                                      President and CEO
                                     (Principal Executive Officer)




  DATE: June 17, 1997                 By: /s/ Marvin G. Kiser, Sr.
       ------------------                 ------------------------
                                      Marvin G. Kiser, Sr.
                                     (Principal Accounting Officer)





                                       10