<PAGE>   1
                                                                      EXHIBIT 11

HANOVER DIRECT, INC.
COMPUTATION OF EARNINGS (LOSS) PER SHARE

<TABLE>
<CAPTION>
                                                 (In 000's, except per share amounts)

                                                           13 WEEKS ENDED        
                                                   SEPTEMBER 28,     SEPTEMBER 30,
                                                       1996              1995
                                                   ------------        --------
<S>                                                <C>                 <C>      
Net income (loss) before extraordinary item        $    (28,431)       $ (7,749)
Extraordinary item                                       (1,134)         (1,834)
                                                   ------------        --------
Net income (loss)                                       (29,565)         (9,586)
Preferred stock dividends                                   (59)            (66)
                                                   ------------        --------
Net income (loss) applicable to common
shareholders                                       $    (29,624)       $ (9,652)
                                                   ============        ========

Average shares of common stock
    outstanding during the period                       114,252          93,005

Total shares used to calculate PEPS*                    114,252          93,005

Primary earnings per share:

   Earnings (loss) before extraordinary item       $      (0.25)       $  (0.08)
   Extraordinary item                                     (0.01)          (0.02)
                                                   ------------        --------
   Primary earnings per share                      $      (0.26)       $  (0.10)
                                                   ============        ========

Average shares of common stock
    outstanding during the period                       114,252          93,005

Total shares used to calculate FDEPS*                   114,252          93,005

Fully diluted earnings per share:
    Earnings (loss) before extraordinary
    item                                           $      (0.25)       $  (0.08)
    Extraordinary item                                    (0.01)          (0.02)
                                                   ------------        --------
    Fully diluted earnings per share               $      (0.26)       $  (0.10)
                                                   ============        ========

Average shares of common stock
outstanding during the period                           114,252          93,005
Basic earnings per share:
    Earnings (loss) before extraordinary
    item                                           $      (0.25)       $  (0.08)
    Extraordinary item                                    (0.01)          (0.02)
                                                   ------------        --------
    Basic earnings per share                       $      (0.26)       $  (0.10)
                                                   ============        ========
</TABLE>


--------------------------------------

* Per APB 15, when a net loss is reported, exercise or conversion of stock
options is not to be assumed.



                                       27