<PAGE>   1

                       SECURITIES AND EXCHANGE COMMISSION

                             WASHINGTON, D.C. 20549

                                   FORM 10-Q

                   QUARTERLY REPORT UNDER SECTION 13 OR 15(d)
                     OF THE SECURITIES EXCHANGE ACT OF 1934
                     --------------------------------------

                   FOR THE QUARTERLY PERIOD ENDED MAY 3, 1997
                         COMMISSION FILE NUMBER 0-19714

                                PERFUMANIA, INC.

           STATE OF FLORIDA                           I.R.S. NO. 65-0026340

                             11701 N.W. 101st ROAD
                              MIAMI, FLORIDA 33178

                        TELEPHONE NUMBER: (305) 889-1600

INDICATE BY CHECK MARK WHETHER THE REGISTRANT, (1) HAS FILED ALL REPORTS
REQUIRED TO BE FILED BY SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF
1934 DURING THE PRECEDING TWELVE (12) MONTHS (OR FOR SUCH SHORTER PERIOD THAT
THE REGISTRANT WAS REQUIRED TO FILE SUCH REPORTS) AND (2) HAS BEEN SUBJECT TO
SUCH FILING REQUIREMENTS FOR THE PAST NINETY (90) DAYS.

                                 YES  X      NO
                                    -----      -----

                          COMMON STOCK $.01 PAR VALUE
                 OUTSTANDING SHARES AT MAY 3, 1997 - 7,807,791


<PAGE>   2


                               TABLE OF CONTENTS

                                PERFUMANIA, INC.

                                     PART I
                             FINANCIAL INFORMATION

ITEM 1        FINANCIAL STATEMENTS

                  Consolidated Balance Sheets................................ 3

                  Consolidated Statements of Operations...................... 4

                  Consolidated Statements of Cash Flows...................... 5

                  Notes to Condensed Consolidated Financial Statements....... 6


ITEM 2        MANAGEMENT'S DISCUSSION AND ANALYSIS OF
               CONSOLIDATED FINANCIAL CONDITION AND
               RESULTS OF OPERATIONS......................................... 8


                                       2
<PAGE>   3



PART I.  FINANCIAL INFORMATION
ITEM 1.  FINANCIAL STATEMENTS
<TABLE>
<CAPTION>

                                PERFUMANIA, INC.
                          CONSOLIDATED BALANCE SHEETS

                                                          May 3, 1997   February 1, 1997
                                                          -----------   ----------------
<S>                                                      <C>              <C>
ASSETS

Current assets:
  Cash and cash equivalents                              $   1,391,835    $   1,641,527
  Trade receivables, net:
    Customers, less allowance for doubtful accounts of
       $283,386 and $248,386                                 8,752,319       12,275,159
    Affiliates                                               1,175,696          653,657
  Advances to suppliers                                      6,903,210        5,023,718
  Inventories, net of reserve of $940,000                   86,881,847       85,110,423
  Prepaid expenses and other current assets                  1,449,511        1,899,320
  Net deferred tax asset                                     2,232,627          873,472
  Due from related parties                                      85,613           85,613
                                                         -------------    -------------
     Total current assets                                  108,872,658      107,562,889
Property and equipment, net                                 18,189,577       17,709,758
Leased equipment under capital leases, net                   1,908,791        2,013,857
Other assets                                                 2,214,937        2,203,442
Due from related parties                                       457,243          417,763
                                                         -------------    -------------
                                                         $ 131,643,206    $ 129,907,709
                                                         =============    =============
LIABILITIES AND STOCKHOLDERS' EQUITY

Current liabilities:
  Bank line of credit and current portion of
     notes payable                                       $  36,138,200    $  31,372,171
  Accounts payable                                          37,109,701       36,128,515
  Accrued expenses and other liabilities                     3,018,360        3,940,440
  Income taxes payable                                         339,203        1,321,203
  Current portion of obligations under capital leases          630,979          873,425
  Due to related parties                                       754,483          770,000
                                                         -------------    -------------
     Total current liabilities                              77,990,926       74,405,754
Long term portion of loans payable                           4,718,487        4,519,859
Long-term portion of obligations under capital leases        1,187,516        1,187,516
                                                         -------------    -------------
     Total liabilities                                      83,896,929       80,113,129
                                                         -------------    -------------
Excess of fair value of assets acquired over cost              616,169          470,000
                                                         -------------    -------------

Stockholders' equity:
  Preferred stock, $.01 par value, 1,000,000 shares
   authorized, none issued                                          --               --
  Common stock, $.01 par value, 25,000,000 shares
   authorized, 7,807,791 shares issued
   and outstanding                                              78,078           78,078
  Capital in excess of par                                  51,900,229       51,900,229
  Treasury stock                                            (2,810,849)      (2,655,110)
  Retained earnings (accumulated deficit)                   (2,037,350)           1,383
                                                         -------------    -------------
Total stockholders' equity                                  47,130,108       49,324,580
                                                         -------------    -------------
                                                         $ 131,643,206    $ 129,907,709
                                                         =============    =============


</TABLE>

See accompanying notes to consolidated financial statements.

                                       3
<PAGE>   4

                                PERFUMANIA, INC.
                     CONSOLIDATED STATEMENTS OF OPERATIONS

                                              Thirteen        Thirteen
                                             Weeks Ended     Weeks Ended
                                             May 3, 1997     May 4, 1996
                                             -----------     -----------

Net sales:
   Unaffiliated customers                    $28,946,685     $23,220,473
   Affiliates                                  1,042,039              --
                                             -----------     -----------
                                              29,988,724      23,220,473
                                             -----------     -----------

Cost of goods sold:
   Unaffiliated customers                     16,040,684      13,312,526
   Affiliates                                  1,042,039              --
                                             -----------     -----------
                                              17,082,723      13,312,526
                                             -----------     -----------
     Gross profit                             12,906,001       9,907,947
                                             -----------     -----------

Operating expenses:
   Selling, general and administrative        14,111,897      10,278,919
   Depreciation and amortization               1,294,004         854,190
                                             -----------     -----------
      Total operating expenses                15,405,901      11,133,109
                                             -----------     -----------


Loss from operations before other expense     (2,499,900)     (1,225,162)
Other expense                                   (897,988)       (724,989)
                                             -----------     -----------
Loss before income taxes                      (3,397,888)     (1,950,151)
Benefit for income taxes                      (1,359,155)       (760,559)
                                             -----------     -----------
Net loss                                     ($2,038,733)    ($1,189,592)
                                             ===========     ===========

Loss per common share                             ($0.28)         ($0.16)
                                             ===========     ===========



See accompanying notes to consolidated financial statements.

                                       4
<PAGE>   5


                              PERFUMANIA, INC.
                 CONSOLIDATED STATEMENTS OF CASH FLOWS

<TABLE>
<CAPTION>

                                                                 Thirteen       Thirteen
                                                               Weeks Ended     Weeks Ended
                                                               May 3, 1997     May 4, 1996
                                                               -----------     -----------
<S>                                                             <C>             <C>
Cash flows from operating activities:
  Net loss                                                      ($2,038,733)   ($1,189,592)
  Adjustments to reconcile net loss to net cash
    used in operating activities:
    Capitalized from preopening costs                               (96,927)      (149,489)
    Provision for doubtful accounts                                  35,000         60,000
    Deferred tax benefit                                         (1,359,155)      (760,559)
    Depreciation and amortization                                 1,294,004        854,190
    Loss on disposition                                             115,000         16,319
    Change in assets and liabilities, (Increase) decrease in:
        Trade receivables                                         2,965,801      3,658,950
        Advances to suppliers                                    (1,879,492)    (2,248,438)
        Inventories                                              (1,771,424)    (8,021,023)
        Other current assets                                        410,329         54,986
        Other assets                                                (79,764)        29,540
    Increase (decrease) in:
        Accounts payable                                            982,000      2,486,015
        Accrued expenses and other current liabilities             (890,911)       (40,745)
        Income taxes payable                                       (982,000)            --
                                                                -----------    -----------
    Total adjustments                                            (1,258,353)    (4,060,254)
                                                                -----------    -----------
      Net cash used in operating activities                      (3,297,086)    (5,249,846)
                                                                -----------    -----------
Cash flows from investing activities:
    Additions to property and equipment                          (1,495,595)    (1,268,042)
                                                                -----------    -----------
      Net cash used in investing activities                      (1,495,595)    (1,268,042)
                                                                -----------    -----------
Cash flows from financing activities:
   Net borrowings and repayments under bank
      line of credit and notes payable                            4,964,657      3,183,935
   Issuance of convertible debentures                                  --        2,935,361
   Net repayment to related parties                                 (15,517)        (2,236)
   Prinicipal payments under capital lease obligations             (250,412)      (118,462)
   Proceeds from issuance of common stock                              --            8,250
   Purchases of treasury stock                                     (155,739)          --
   Subscription of stock                                               --          956,250
                                                                -----------    -----------
      Net cash provided by financing activities                   4,542,989      6,963,098
                                                                -----------    -----------
Increase (decrease) in cash and cash equivalents                   (249,692)       445,210
Cash and cash equivalents at beginning of period                  1,641,527        331,028
                                                                -----------    -----------
Cash and cash equivalents at end of period                      $ 1,391,835    $   776,238
                                                                ===========    ===========

Supplemental disclosure of cash flow information:
-------------------------------------------------

     Cash paid for:
        Interest                                                $   896,244    $   797,547
        Income Taxes                                                982,000         25,000

</TABLE>

See accompanying notes to consolidated financial statements.

                                       5
<PAGE>   6




                                PERFUMANIA, INC.
              NOTES TO CONDENSED CONSOLIDATED FINANCIAL STATEMENTS

(1).  SIGNIFICANT ACCOUNTING POLICIES

The condensed consolidated financial statements include the accounts of
Perfumania and subsidiaries (the Company). All material intercompany balances
and transactions have been eliminated in consolidation.

The unaudited condensed consolidated financial statements have been prepared
pursuant to the rules and regulations of the Securities and Exchange
Commission. Certain information and note disclosures normally included in
annual financial statements prepared in accordance with generally accepted
accounting principles have been condensed or omitted pursuant to those rules
and regulations, although the Company believes that the disclosures made are
adequate to make the information presented not misleading. The financial
information presented herein, which is not necessarily indicative of results to
be expected for the current fiscal year, reflects all adjustments which, in the
opinion of the Company, are necessary for a fair statement of the results for
the periods indicated. It is suggested that these condensed consolidated
financial statements be read in conjunction with the financial statements and
the notes thereto included in the Company's Annual Report on Form 10-K for the
fiscal year ended February 1, 1997.

(2).  STOCKHOLDERS' EQUITY

<TABLE>
<CAPTION>

                                                    Capital
                            Common Stock           In Excess             Treasury Stock          Retained
                    ---------------------------     of Par       ---------------------------     Earnings
                       Shares         Amount         Value          Shares         Amount        (Deficit)          Total
                    ------------   ------------   ------------   ------------   ------------    ------------    ------------
<S>                    <C>         <C>            <C>                 <C>       <C>             <C>             <C>
Balance at
February 1, 1997       7,807,791   $     78,078   $ 51,900,229        667,900   ($ 2,655,110)   $      1,383    $ 49,324,580

Purchases of
treasury stock                --             --             --         45,400       (155,739)             --        (155,739)

Net loss for the
thirteen weeks
ended May 3, 1997             --             --             --             --             --      (2,038,733)     (2,038,733)
                    ------------   ------------   ------------   ------------   ------------    ------------    ------------

Balance at
May 3, 1997            7,195,983   $     78,078   $ 51,900,229        713,300    ($2,810,849)    ($2,037,350)   $ 47,130,108
                    ------------   ------------   ------------   ------------   ------------    ------------    ------------


</TABLE>

(3).  EARNINGS (LOSS) PER COMMON SHARE

Earnings (loss) per common share are computed by dividing net income (loss) by
the weighted average number of common shares outstanding.

The weighted average number of common shares for the period ended May 3, 1997
and May 4, 1996 were 7,195,983 and 7,489,818.


                                       6
<PAGE>   7



4).  SEGMENT INFORMATION

The Company operates in two industry segments, specialty retail sale and
wholesale distribution of fragrances and related products. Financial
information for these segments is summarized in the following table.

                                              Thirteen Weeks    Thirteen Weeks
                                                 Ended               Ended
                                              May 3, 1997         May 4, 1996
                                              -----------         -----------
Sales
     Wholesale                                $ 6,541,323        $ 5,028,257
     Retail                                    23,447,401         18,192,216
                                              -----------        -----------
          Total net sales                     $29,988,724        $23,220,473
                                              -----------        -----------

Cost of goods sold
     Wholesale                                $ 5,084,973        $ 3,839,397
     Retail                                    11,997,750          9,473,129
                                              -----------        -----------
          Total cost of goods sold            $17,082,723        $13,312,526
                                              -----------        -----------

Gross profit
     Wholesale                                $ 1,456,350        $ 1,188,860
     Retail                                    11,449,651          8,719,087
                                              -----------        -----------
          Total gross profit                  $12,906,001        $ 9,907,947
                                              -----------        -----------

Number of stores                                      270                195



                                              May 3, 1997     February 1, 1997
                                              -----------     ----------------

Inventory
---------
    Wholesale                                 $33,954,792        $32,051,346
    Retail                                     52,927,055         53,059,077
                                              -----------        -----------
                                              $86,881,847        $85,110,423
                                              -----------        -----------

An unaffiliated customer of the wholesale segment accounted for approximately
6% and 9% of the consolidated net sales for the thirteen weeks ended May 3,
1997 and May 4, 1996, respectively, and 59% and 72% of the consolidated net
trade accounts receivable balance at May 3, 1997 and February 1, 1997,
respectively.

                                       7
<PAGE>   8



PART I.  FINANCIAL INFORMATION

ITEM 2.  MANAGEMENT'S DISCUSSION AND ANALYSIS OF
         CONSOLIDATED FINANCIAL CONDITION AND RESULTS OF
         OPERATIONS

SEASONALITY

The Company's operations have historically been seasonal, with generally higher
retail sales in the third and fourth fiscal quarters than in the first and
second fiscal quarters. Significantly higher fourth fiscal quarter retail sales
result from increased purchases of fragrances as gift items during the
Christmas holiday season. Wholesale sales also vary by fiscal quarter as a
result of the selection of merchandise available for sale as well as the need
for the Company to stock its retail stores for the Christmas holiday season.
Therefore, the results of any interim period are not necessarily indicative of
the results that might be expected during a full fiscal year.

LIQUIDITY AND CAPITAL RESOURCES

At May 3, 1997 working capital was $30.9 million compared to $33.2 million at
February 1, 1997. The decrease was primarily due to the current period loss.

Net cash used in operating activities during the thirteen weeks ended May 3,
1997 was approximately $3.3 million, principally as a result of the net change
in the Company's trade receivables, advances to suppliers, inventories and
accounts payable. At May 3, 1997, approximately $0.7 million of the Company's
trade receivables were considered past due compared to $0.2 million at February
1, 1997. Of the $9.9 million in trade receivables, $6.1 million was due from
one customer which also accounted for 27.1% of the Company's wholesale sales
during the thirteen weeks ended May 3, 1997. The Company's sales to this
customer are made on an open account terms and since late 1991 the Company has
extended credit terms to this customer of up to one year. The Company has not
experienced any write-offs of accounts receivable from this customer due to
collectibility.

Net cash used in investing activities during the current period was $1.5
million. This represents purchases of furniture, fixtures and equipment for
store openings and renovations of existing stores during the first and second
quarters.

Net cash provided by financing activities during the current period was
approximately $4.5 million, which was primarily the result of an increase in
the Company's use of its line of credit. On April 16, 1997, the Company's $30
million line of credit, which expires on April 30, 1999, was increased to $35
million.

During the thirteen weeks ended May 3, 1997, the Company opened nine stores and
closed one store. At May 3, 1997, the Company operated 270 stores. The Company
plans to open approximately 40 stores during the remainder of fiscal 1997.

                                       8
<PAGE>   9


RESULTS OF OPERATIONS

Comparison of the Thirteen Weeks Ended May 3, 1997 with the Thirteen Weeks
Ended May 4, 1996.

Net sales increased 29.1% from $23.2 million in the first thirteen weeks of
1997 to $30.0 million in the first thirteen weeks of 1997. The increase in net
sales was the result of a 30.1% increase in wholesale sales (from $5.0 million
to $6.5 million), and a 28.9% increase in retail sales (from $18.2 million to
$23.4 million). The increase in wholesale sales was primarily due to a
difficult wholesale market last year. The increase in retail sales was
principally due to the increase in the number of stores operated during the
first thirteen weeks of 1997 compared to the first thirteen weeks of 1996.
Comparable store sales during the current period increased 4.5% when compared
to last year.

Gross profit increased 30.3% from $9.9 million in the first thirteen weeks of
1996 (42.7% of total net sales) to $12.9 million in the first thirteen weeks of
1997 (43.0% of net sales) primarily due to an increase in gross profit for both
the retail and wholesale divisions.

Gross profit for the wholesale division increased from $1.2 million in the
first thirteen weeks of 1996 to $1.5 million in the first thirteen weeks of
1997 as a result of higher wholesale sales. As a percentage of net sales, gross
profit for the wholesale division decreased from 23.6% in the first thirteen
weeks of 1996 to 22.3% in the first thirteen weeks of 1997.

Gross profit for the retail division increased to $11.4 million in the first
thirteen weeks of 1997 from $8.7 million in the first thirteen weeks of 1996 as
a result of higher retail sales. As a percentage of net sales, gross profit for
the retail division increased from 47.9% in the first thirteen weeks of 1996 to
48.8% in the thirteen weeks of 1997 primarily as a result of less promotional
sales of merchandise at lower margins.

Operating expenses, which include selling, general and administrative expenses
as well as depreciation, increased 38.6% from $11.1 million in the first
thirteen weeks of 1996 to $15.4 million in the first thirteen weeks of 1997.
The increase was primarily due to costs associated with the operation of 73
additional stores during the current period.

The Company had a net loss of $2,038,733, or $0.28 per share, in the first
thirteen weeks of 1997 compared to a net loss of $1,189,592 or $0.16 per share,
in the first thirteen weeks of 1996. The weighted average number of common
shares outstanding were 7,195,983 for the first thirteen weeks of 1997 and
7,489,818 for the first thirteen weeks of 1996.

                                       9
<PAGE>   10


                                PERFUMANIA, INC.

                                   SIGNATURES

Pursuant to the requirements of the Securities Exchange Act of 1934, the
registrant has duly caused the report to be signed on its behalf by the
undersigned, thereunto duly authorized.
 
                                      Perfumania, Inc.
                                      -----------------------------------------
                                      (Registrant)




Date: June 13, 1997               By: /s/ SIMON FALIC
                                      -----------------------------------------
                                      Simon Falic
                                      Chairman of the Board and
                                      Chief Executive Officer
                                      (Principal Executive Officer)




                                  By: /s/ RON A. FRIEDMAN
                                      -----------------------------------------
                                      Ron A. Friedman
                                      President, Chief Financial Officer,
                                      Treasurer and Secretary
                                      (Principal Financial and
                                      Accounting Officer)


                                      10