<PAGE>   1
                                 UNITED STATES
                       SECURITIES AND EXCHANGE COMMISSION
                             WASHINGTON, D.C. 20549

                                   FORM 10-Q


{x} Quarterly Report Pursuant to Section 13 or 15 (d) of the Securities
    Exchange Act of 1934

FOR THE QUARTER ENDED NOVEMBER 2, 1996

                                       OR

( ) Transition Report Pursuant to Section 13 or 15(d) of the Securities
    Exchange Act of 1934

For the transition period from                     to
                               --------------------   ------------------------  

COMMISSION FILE NUMBER 1-8578

                             MCRAE INDUSTRIES, INC
             (Exact name of registrant as specified in its charter)

               DELAWARE                                   56-0706710
     (State or other jurisdiction of                   (I.R.S. Employer
     incorporation or organization)                    Identification No.)

                             402 NORTH MAIN STREET
                        MT. GILEAD, NORTH CAROLINA 27306
                    (Address of principal executive offices)

                         TELEPHONE NUMBER (910)439-6147
              (Registrant's telephone number, including area code)


Indicate by check mark whether the registrant (1) has filed all reports
required to be filed by Section 13 or 15 (d) of the Securities Exchange Act of
1934 during the preceding 12 months (or for such shorter periods that the
registrant was required to file such reports), and (2) has been subject to such
filing requirements for the past 90 days.

     Yes  X                                   No
         ---                                     ---

Indicate the number of shares outstanding of each of the issuer's classes of
common stock, as of the latest practical date.

Common Stock, $l Par Value--Class A 1,809,964 shares as of December 9, 1996
Common Stock, $1 Par Value--Class B   958,535 shares as of December 9, 1996




                                      1

<PAGE>   2


                    MCRAE INDUSTRIES, INC. AND SUBSIDIARIES

                                     INDEX


                                                         Page No.


                         PART 1. FINANCIAL INFORMATION



<TABLE>
ITEM 1.  CONDENSED CONSOLIDATED FINANCIAL STATEMENTS
<S>      <C>                                                <C>
         Condensed Consolidated Balance Sheet                 3

         Condensed Consolidated Statement of Operations       5

         Condensed Consolidated Statement of Cash Flows       6

         Notes to Condensed Consolidated Financial Statements 7



ITEM 2.  MANAGEMENT'S DISCUSSION AND ANALYSIS OF FINANCIAL
          CONDITION AND RESULTS OF OPERATIONS                8-9
</TABLE>





                           PART II. OTHER INFORMATION



<TABLE>
<S>      <C>                                                <C>
ITEM 1.  LEGAL PROCEEDINGS                                  9


ITEM 6.  EXHIBITS AND REPORTS ON FORM 8-K                   9


         SIGNATURES                                         9
</TABLE>






                                      2
<PAGE>   3











PART I.    FINANCIAL INFORMATION

ITEM 1. CONDENSED CONSOLIDATED FINANCIAL STATEMENTS


                    MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                      CONDENSED CONSOLIDATED BALANCE SHEET
                                   ASSETS
               (In thousands, except share and per share data)

<TABLE>
<CAPTION>
                                            November 2, 1996     August 3, 1996
Assets                                        (Unaudited)            (Note)
                                            ----------------     --------------
Current assets:
     <S>                                        <C>                <C>  
     Cash and cash equivalents                  $ 3,391            $   581

     Securities                                      65                 65

     Accounts and notes receivable,net            9,370             10,606

     Inventories                                 12,586             12,640

     Net investment in capitalized leases           956                966

     Prepaid expenses and other current assets      168                210
                                                -------            -------

       Total current assets                      26,536             25,068
                                                -------            -------

Property, plant and equipment,net                 7,278              7,172


Other assets:

     Receivables,related entities                 2,477              2,359
                                                
     Net investment in capitalized leases         1,610              1,798
                                                
     Notes receivable                             1,004                952
                                                
     Real estate held for investment                478                478
                                                
                                                
     Goodwill                                       664                674
                                                
     Other                                        1,083              1,060
                                                -------            -------
       Total other assets                         7,316              7,321
                                                -------            -------

                                                $41,130            $39,561
                                                =======            =======
</TABLE>




                  See notes to quarterly financial statements


                                      3

<PAGE>   4


                    MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                      CONDENSED CONSOLIDATED BALANCE SHEET
                    LIABILITIES AND SHAREHOLDERS' EQUITY
               (In thousands, except share and per share data)


<TABLE>
<CAPTION>
                                             November 2, 1996    August 3, 1996
                                               (Unaudited)           (Note)
                                             ----------------    --------------
Current liabilities:                                               
     <S>                                          <C>              <C>
     Notes payable to bank                        $   721          $   789
                                                                 
     Accounts payable                               3,643            2,897
                                                                 
     Accrued employee benefits                        885              846
                                                                 
     Deferred revenues                              1,436            1,454
                                                                 
     Accrued payroll and payroll taxes                631              790
                                                                 
     Income taxes                                     994              759
                                                                 
     Other                                            598              580
                                                  -------          -------
                                                                 
       Total current liabilities                    8,908            8,115
                                                  -------          -------
                                                                 
Notes payable to banks, net of current portion      6,085            6,285
                                                                 
Minority Interest                                     824              797
                                                                 
                                                                 
Shareholders' Equity:                                            
   Common stock:                                               
     Class "A", $1 par; Authorized 5,000,000                     
       shares; Issued and outstanding, 1,802,786                   
       and 1,788,286, shares, respectively          1,803            1,788
     Class "B", $1 par, Authorized 2,500,000                     
       shares; Issued and outstanding 965,713                      
       and 951,213 shares, respectively               966              951
                                                                 
     Additional Paid-in Capital                       734              705
                                                                 
     Retained Earnings                             21,810           20,920
                                                  -------          -------
       Total Shareholders' Equity                  25,313           24,364
                                                  -------          -------
                                                  $41,130          $39,561
                                                  =======          =======
</TABLE>

NOTE - The balance sheet at August 3, 1996 has been derived from the audited
financial statements at that date but does not include all of the information
and footnotes required by generally accepted accounting principles for complete
financial statements.




                  See notes to quarterly financial statements


                                      4

<PAGE>   5

                    MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                 CONDENSED CONSOLIDATED STATEMENT OF OPERATIONS
                (In thousands, except share and per share data)
                                  (Unaudited)


<TABLE>
<CAPTION>
                                                Three Months Ended
                                       November 2, 1996    October 28, 1995
                                       ------------------------------------
<S>                                         <C>                  <C>
Net revenues                                $   17,141           $   10,412

     Costs and expenses:

     Cost of revenues                           12,579                7,343

     Selling and administrative                  3,008                2,536

     Other expense (income), net                    51                  (75)
                                            ----------           ----------

Total costs and expenses                        15,638                9,804
                                            ----------           ----------

Earnings before income taxes
     and minority interest                       1,503                  608

Provision for income taxes                         587                  229

Minority shareholder's interest
     in earnings of subsidiary                      26                   35
                                            ----------           ----------

Net earnings                                $      890           $      344
                                            ==========           ==========

Net earnings per Common Share               $      .32           $      .13
                                            ----------           ----------
Weighted average number of Common
     Shares Outstanding                      2,742,047            2,731,507
                                            ----------           ----------
</TABLE>















                  See notes to quarterly financial statements.

                                      5

<PAGE>   6


                    MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
                 CONDENSED CONSOLIDATED STATEMENT OF CASH FLOWS
                                 (In thousands)
                                  (Unaudited)

<TABLE>
<CAPTION>
                                                Three Months Ended
                                       November 2, 1996    October 28, 1995
                                       ------------------------------------
<S>                                               <C>           <C>


Net cash provided by (used in) operating
     activities                                   $  3,289        (291)
                                                                
Cash flows from investing activities:                           
                                                                
     Proceeds from (purchase of) securities            -0-        1304
                                                                
     Capital expenditures                              (93)       (270)
                                                                
     Purchase of other assets                          -0-         (57)
                                                                
     Net (advances) payments of                                 
      long term receivables                           (176)        (54)
                                                  ---------     ------
                                                                
Net cash provided by (used in)                                  
      investing activities                            (269)        923
                                                                
Cash flows from financing activities:                           
                                                                
     Principal repayments of notes payable            (268)        (15)
                                                                
     Purchase Treasury Stock                           -0-         (11)
                                                                
     Proceeds from exercise of stock options            58         -0-
                                                                
     Dividends paid                                    -0-        (155)
                                                  ---------     ------
                                                                
Net cash used in financing activities                 (210)       (181)
                                                  ---------     ------
                                                                
Net increase in cash and cash equivalents            2,810         451
                                                                
Cash and Cash Equivalents at Beginning of Period       581         628
                                                  ---------     ------
                                                                
Cash and Cash Equivalents at End of Period        $  3,391      $1,079
                                                  ========      ======
</TABLE>







                  See notes to quarterly financial statements.

                                      6


<PAGE>   7


                    MCRAE INDUSTRIES, INC. AND SUBSIDIARIES
              NOTES TO CONDENSED CONSOLIDATED FINANCIAL STATEMENTS
                                 (UNAUDITED)



NOTE A - BASIS OF PRESENTATION

The accompanying unaudited consolidated financial statements have been prepared
in accordance with generally accepted accounting principles for interim
financial information and with the instructions to Form 10-Q and Article 10
Regulation S-X. Accordingly, they do not include all of the information and
footnotes required by generally accepted accounting principles for complete
financial statements. In the opinion of management, all adjustments (consisting
of normal recurring accruals) considered necessary for a fair presentation have
been included. Operating results for the three months ended November 2, 1996
are not necessarily indicative of the results that may be expected for the year
ended August 2, 1997. For further information, refer to the consolidated
financial statements and footnotes thereto included in the McRae Industries,
Inc. annual report on Form 10-K for the year ended August 3, 1996.

Certain reclassifications have been made to the prior year's financial
statements to conform with the current year's presentation.


NOTE B - INVENTORIES

An actual valuation of inventory under the LIFO method can be made only at the
end of each year based on the inventory levels and costs at that time.
Accordingly, interim LIFO calculations must necessarily be based on
management's estimates of expected year-end inventory levels and costs. Because
these are subject to many forces beyond management's control, interim
calculations, if any, are subject to the final year-end LIFO inventory
valuation.

The components of inventory consist of the following (in thousands):

<TABLE>
<CAPTION>
                                  November 2, 1996        August 3, 1996
                                  --------------------------------------        
<S>                                   <C>                     <C>
Raw materials                         $ 2,074                 $ 2,288
Work in process                         1,006                   1,142
Finished goods                          9,506                   9,210
                                      -------------------------------   
                                      $12,586                 $12,640
                                      ===============================
</TABLE>        


NOTE C - SUBSEQUENT EVENTS

On November 26, 1996, the Company declared a cash dividend of 9.0 cents per
share on its Class A Common Stock, payable to shareholders of record on
December 6, 1996 on December 23, 1996.






                                      7

<PAGE>   8




                    MCRAE INDUSTRIES, INC. AND SUBSIDIARIES


ITEM 2. MANAGEMENT'S DISCUSSION AND ANALYSIS OF FINANCIAL CONDITION AND
        RESULTS OF OPERATIONS

FINANCIAL CONDITION

For the quarter ended November 2, the Company's balance sheet remained solid
with a current ratio of almost 3 to 1. Total assets grew by $1.6 million, a
quarterly increase of approximately 4%.

Operating activities provided net cash of $3.3 million. Net income from
operations accounted for 27% of this total while depreciation and amortization
contributed 12%. Collection of government receivables as well as other notes
receivable provided 39% or $1.3 million in net cash. Inventories used $354,000
primarily as a result of the purchase of additional rental equipment for the
office products segment. The normal timing of processing and payment procedures
increased accounts payable and income taxes payable. Net cash provided by these
items amounted to $883,000 or 27% of the net cash generated from operating
activities.

Investing activites recorded a net cash usage of $269,000. Capital expenditures
for mostly production machinery and equipment and computer hardware and
software accounted for 35% of this total usage while additional advances to
certain related parties accounted for the remaining $179,000.

Principal repayments of notes payable used $268,000 and was partially offset by
proceeds from the exercise of employee stock options. As a result, financing
activities resulted in a net cash usage of $210,000.

The Company continues to carry lines of credit with several banks totalling
$3.75 million. At the end of the quarter, $400,000 had been utilized.
Management believes that the current cash and cash equivalents, available lines
of credit and cash flows from operations will be sufficient to support future
working capital and debt repayment requirements.


FIRST QUARTER FISCAL 1997 COMPARED TO FIRST QUARTER FISCAL 1996

Consolidated revenues for the first quarter of 1997 increased 65% or $6.72
million over the first quarter of fiscal 1996. The Footwear segment was
primarily responsible for $6.65 million of the increase. The military combat
boot unit exceeded last year's first quarter by 97% while the inclusion of the
western and work boot unit, American West Trading Company (American West),
contributed $4.47 million. American West was purchased on April 30, 1996 and,
as a result, had no contribution for the first quarter of fiscal 1996. The
office products segment also recorded an increase over the same period of the
prior year of almost $387,000. Revenues for the bar code unit declined 6% for
the current fiscal quarter as compared to the same period last year.

Gross profit for the first quarter of fiscal 1997 amounted to $4.6 million
compared to $3.1 million for the same period in fiscal 1996, a 48% increase.
This increase is primarily attributable to the increase in sales of military
combat boots and the inclusion of American West. Each of the major operating
segments of the Company reported slight improvements in their respective
operating margins; however, gross profit as a percentage of sales for the
Company as a whole declined to 26.6% from 29.5% in the comparable quarters of
1997 and 1996, respectively. This decline resulted from the increased mix of
sales from the footwear segment which is characterized by lower margins. The
footwear segment contributed 52% of consolidated revenues for the first quarter
in 1997 compared to only 22% for the same period in 1996.


                                      8

<PAGE>   9

Selling, general and administrative expenses (SG&A) increased by $470,000 or
19% for the first quarter of fiscal 1997 as compared to the first quarter of
fiscal 1996. Again, the SG&A expenses related to the western and work boot unit
was the major contributor to the comparative quarterly increase.


PART II. OTHER INFORMATION

ITEM 1. LEGAL PROCEEDINGS

Reference is made to Item 3 of the Company's Annual Report to Shareholders on
Form 10-K for the fiscal year ended August 3, 1996.

ITEMS 2 THROUGH 5.

These items are inapplicable and have been omitted.

ITEM 6. EXHIBITS AND REPORTS ON FORM 8-K

(a)  Exhibits

     Exhibit 27   Financial Data Schedule. (Filed in electronic format only.
                  Pursuant to Rule 402 of Regulations S-T, this schedule shall
                  not be deemed filed for purposes of Section 11 of the
                  Securities Act of 1933 or Section 18 of the Securities
                  Exchange Act of 1934.)

(b) No reports on Form 8-K were filed during the quarter ended November 2,
    1996.



SIGNATURES

Pursuant to the requirements of the Securities Exchange Act of 1934, the
registrant has duly caused this report to be signed on its behalf by the
undersigned thereunto duly authorized.


                                                McRae Industries, Inc.
                                                     (Registrant)




DATE: December 13, 1996                         By: /s/ B.J. McRae
                                                    ------------------------
                                                B.J. McRae
                                                President
                                                (Principal Executive Officer)




DATE: December 13, 1996                         By: /s/ Marvin G. Kiser, Sr.
                                                    ------------------------
                                                Marvin G. Kiser, Sr.
                                                (Principal Accounting Officer)





                                      9