<PAGE>   1
                       SECURITIES AND EXCHANGE COMMISSION
                             WASHINGTON, D.C. 20549
                                    FORM 10-Q

/X/   QUARTERLY REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES
EXCHANGE ACT OF 1934

      For the quarterly period ended September 30, 1996

/   / TRANSITION REPORT PURSUANT OR TO SECTION 13 or 15(d) OF THE SECURITIES
EXCHANGE ACT OF 1934

      For the transition period __________ to ___________.

                         Commission File Number: 1-10916

                             INTERVISUAL BOOKS, INC.
             (Exact name of registrant as specified in its charter)

            California                                      95-2929217
(State or other jurisdiction of                          (I.R.S. Employer
incorporation or organization)                          Identification No.)

2850 Ocean Park Boulevard, Suite 225
Santa Monica, California                                  90405
Address of principal executive offices                   Zip Code

Registrant's telephone number, including area code:   (310)  396-8708


------------------------------------------------------------------------
Former name, former address and former fiscal year, if changed since last
report.

      Indicate by check whether the registrant (1) has filed all reports
required to be filed by Section 13 or 15(d) of the Securities Exchange Act of
1934 during the preceding 12 months (or for such shorter period that the
registrant was required to file such reports), and (2) has been subject to such
filing requirements for the past 90 days. Yes X No_____

   APPLICABLE ONLY TO ISSUERS INVOLVED IN BANKRUPTCY PROCEEDINGS DURING THE
                            PRECEDING FIVE YEARS:

      Indicate by check mark whether the registrant has filed all documents and
reports required to be filed by Sections 12, 13 or 15(d) of the Securities
Exchange Act of 1934 subsequent to the distribution of securities under a plan
confirmed by a court. Yes_____ No_____


      APPLICABLE ONLY TO CORPORATE ISSUERS: Indicate the number of shares
outstanding of each of the issuer's classes of common stock, as of the latest
practical date. As of September 30, 1996, there were 4,782,798 shares of common
stock outstanding.
<PAGE>   2
                             INTERVISUAL BOOKS, INC.



                                TABLE OF CONTENTS


PART I - FINANCIAL INFORMATION                                              Page
                                                                            ----
      Item 1.  Financial Statements

            Balance Sheets - September 30, 1996, and December 31, 1995        1

            Statements of Operations - Three months ended September 30,
            1996 and 1995; Nine months ended September 30, 1996 and 1995      2

            Statements of Cash Flows - Nine months ended September 30,
            1996 and 1995                                                     3

            Notes to Financial  Statements - September 30, 1996               4

      Item 2.  Management's Discussion and Analysis of Financial Condition    5
            and Results of Operations

PART II - OTHER INFORMATION

      Item 4.  Submission of Matters to a Vote of Security-Holders            7

      Item 5.  Other Information                                              8

      Item 6.  Exhibits and Reports on Form 8-K                               8

SIGNATURES                                                                    8


                                       i
<PAGE>   3
                             INTERVISUAL BOOKS, INC.
                                 BALANCE SHEETS
                                 (In thousands)


<TABLE>
<CAPTION>
    ASSETS                                                 9/30/96      12/31/95
---------------                                            -------      --------
                                                         (Unaudited)
<S>                                                        <C>           <C>    
Current Assets:
  Cash and cash equivalents                                $ 1,063       $   915
  Investment in marketable securities                        2,348         1,928
  Accounts receivable, less allowances
    of $204 and $158                                         5,574         6,941
  Inventories                                                  374           357
  Prepaid expenses                                             900           541
                                                           -------       -------
    Total Current Assets                                    10,259        10,682

Production costs, net of accumulated
    amortization $12,544 and $11,647                         3,038         3,139

Property and equipment, less accumulated
  depreciation                                                 234           231
                                                           -------       -------
                                                           $13,531       $14,052
                                                           =======       =======
    LIABILITIES AND STOCKHOLDERS' EQUITY
----------------------------------------------
Current Liabilities:
  Accounts payable                                           4,709         5,030
  Accrued royalties                                            592           709
  Accrued expenses                                             181           198
  Income taxes payable                                          --            38
  Customer deposits                                            221            53
                                                           -------       -------
    Total Current Liabilities                                5,703         6,028

Deferred income taxes                                          466           466
                                                           -------       -------
     TOTAL LIABILITIES                                       6,169         6,494
                                                           -------       -------
Stockholders' Equity:
   Common stock, no par value; shares
     authorized 10,000,000, shares issued
     and outstanding 4,782,798 at September 30, 1996
     and December 31, 1995                                   4,044         4,044
   Additional paid in capital                                  209           209
   Retained earnings                                         3,109         3,305
                                                           -------       -------
     TOTAL STOCKHOLDERS' EQUITY                              7,362         7,558
                                                           -------       -------
                                                           $13,531       $14,052
                                                           =======       =======
</TABLE>

See accompanying notes to financial statements.


                                       1
<PAGE>   4
                             INTERVISUAL BOOKS, INC.
                            STATEMENTS OF OPERATIONS
                      (In thousands except per share data)
                                   (Unaudited)


<TABLE>
<CAPTION>
                                               Quarter ended              Nine Months ended
                                                September 30                 September 30
                                              1996         1995          1996            1995
                                             ------       ------       --------        -------
<S>                                          <C>          <C>          <C>             <C>    
Net Sales                                    $5,771       $5,443       $ 12,397        $13,612

Cost of Sales                                 4,474        4,200          9,578         10,603
                                             ------       ------       --------        -------
  Gross Profit                                1,297        1,243          2,819          3,009

Selling, General and Administrative             997          955          3,187          3,042
Expenses
Other Income                                      0          153              0            154
Interest Income, Net                             14           42             71             96
                                             ------       ------       --------        -------
Income (Loss) Before Taxes                      314          483           (297)           217

Income Tax (Benefit)                            107          182           (101)           117
                                             ------       ------       --------        -------
Net Income (Loss)                            $  207       $  301       $   (196)       $   100
                                             ======       ======       ========        =======
Income (Loss) Per Share                      $ 0.04       $ 0.06       $  (0.04)       $  0.02
                                             ======       ======       ========        =======
Weighted Average Number of Common
    Shares and Equivalents Outstanding        4,963        5,016          4,783          4,984
                                             ======       ======       ========        =======
</TABLE>

 
                                        2
<PAGE>   5
                            STATEMENTS OF CASH FLOWS
                 Increase(Decrease) in Cash and Cash Equivalents
                                 (In thousands)

<TABLE>
<CAPTION>
                                                                  Nine Months ended
                                                                     September 30
                                                                  1996           1995
                                                                -------        -------
                                                                      (Unaudited)
<S>                                                             <C>            <C>    
Cash flows from operating activities:
   Net income (loss)                                            $  (196)       $   100

   Adjustments to reconcile net income (loss) to net cash
   provided by operating activities:
       Depreciation and amortization                                953            996
       Provision for losses on accounts receivable                   46             45
       Provision for abandoned titles                                32             32
       Increase (decrease) from changes in:
           Accounts receivable                                    1,321         (1,370)
           Inventories                                              (17)          (108)
           Prepaid expenses                                        (359)          (147)
           Accounts payable                                        (321)         1,414
           Accrued royalties                                       (117)          (104)
           Accrued expenses                                         (17)            (3)
           Income taxes payable                                     (38)          (111)
           Customer deposits                                        168             32
           Deferred revenue                                           0            (60)
                                                                -------        -------
             Net cash provided by operating activities            1,455            716
                                                                -------        -------
Cash flows from investing activities:
   Purchase (sale) of marketable securities                        (420)          (394)
   Additions to property and equipment                              (59)            (9)
   Additions to leasehold improvements                                0             (2)
   Additions to production costs                                   (828)          (957)
                                                                -------        -------
             Net cash used in investing activities               (1,307)        (1,362)
                                                                -------        -------
Net increase (decrease) in cash and cash equivalents                148           (646)

Cash and cash equivalents, beginning of period                      915          2,625
                                                                -------        -------
Cash and cash equivalents, end of period                        $ 1,063        $ 1,979
                                                                =======        =======

Supplemental disclosures of cash flow information:
Cash paid during the period for:
    Interest                                                    $     8        $    --
    Income taxes                                                $    30        $     1
</TABLE>

See accompanying notes to financial statements.


                                       3
<PAGE>   6
                          NOTES TO FINANCIAL STATEMENTS
                               September 30, 1996
                                   (unaudited)

Note 1 - Statement of Information Furnished

In the opinion of management the accompanying unaudited financial statements
contain all adjustments (consisting only of normal and recurring accruals)
necessary to present fairly the financial position as of September 30, 1996, and
the results of operations and cash flows for the three and nine month periods
ended September 30, 1996 and 1995. These results have been determined on the
basis of generally accepted accounting principles and practices applied
consistently with those used in the preparation of the Company's Annual Report
on Form 10-K for the fiscal year ended December 31, 1995.

The results of operations for the three and nine month periods ended September
30, 1996 are not necessarily indicative of the results to be expected for any
other period or for the entire year.

Certain information and footnote disclosures normally included in financial
statements presented in accordance with generally accepted accounting principles
have been condensed or omitted. The accompanying financial statements should be
read in conjunction with the Company's audited financial statements and notes
thereto included in the Company's Annual Report on Form 10-K for the year ended
December 31, 1995.

Note 2 - Earnings Per Share

For the three months ended September 30, 1996 earnings per share amounts are
computed based upon the weighted average number of common shares actually
outstanding during the period plus the additional shares that would be
outstanding assuming exercise of dilutive common stock options and purchase
warrants, which are considered common stock equivalents. The number of shares
that would be issued from the exercise of such options and warrants has been
reduced by the number of common shares that could have been purchased from the
assumed use of the proceeds at the average price of the Company's common stock
during the period (treasury stock method).

For the nine months ended September 30, 1996 earnings per common share amounts
are computed based upon the weighted average number of common shares actually
outstanding during the period. Common stock options and purchase warrants, which
are considered common stock equivalents, are not considered in the average
number of shares as their inclusion would be anti-dilutive.

Fully diluted earnings per share for the three and nine months ended September
30, 1996 do not differ significantly from primary earnings per share.


                                       4
<PAGE>   7
Item 2. Management's Discussion and Analysis of Financial Condition and Results
of Operations

RESULTS OF OPERATIONS

Net sales for the three and nine month periods ended September 30, 1996, were
$5,771,000 and $12,397,000, respectively, as compared to $5,443,000 and
$13,612,000 for the corresponding periods of the preceding year. The sales
increase for the three month period of $328,000 was comprised of an increase of
$731,000 in the sales of backlist titles and a decrease of $403,000 in the sales
of new titles. The sales decrease for the nine month period of $1,215,000
included a decrease of $708,000 in the sales of backlist titles and a decrease
of $507,000 in the sales of new titles. Sales backlog at September 30, 1996, was
$4,628,000 compared to $5,743,000 at September 30, 1995. Foreign sales for the
three and nine month periods ended September 30, 1996, were $2,657,000 and
$4,973,000 or 46% and 40% of total sales, respectively, as compared to
$3,376,000 and $6,760,000 or 62% and 50% of total sales for the corresponding
periods of 1995. Most of the decline in sales can be attributed to reduced
foreign sales, especially in the U.K. and Japan. In the U.K. the book trade has
been adjusting to the elimination of the "net book agreement" which kept
retailers from discounting. As a result some U.K. publishers have been slower in
placing new book and backlist orders with the Company. Japan is implementing a
new tax on books which goes into effect in 1997. There have been delays by the
government in finalizing the actual rate; consequently, Japanese publishers are
delaying orders until the final tax has been determined. Additionally, U.S.
publishers seem to be placing a priority on purchases of the new titles and
reluctant to reorder the backlist titles. Based on orders received to date, the
Company expects that sales for the fourth quarter of 1996 will be less than
sales in the fourth quarter of 1995.

Gross profit margin for the three and nine month periods ended September 30,
1996, was 22.5% and 22.7%, respectively, as compared to 22.8% and 22.1% for the
corresponding periods of 1995. Cost of sales consists primarily of manufacturing
costs, book development amortization and royalties. Manufacturing costs for the
three and nine month periods were $3,803,000, or 65.9% of sales, and $8,113,000
or 65.4% of sales, respectively, as compared to $3,659,000, or 67.2% of sales,
and $9,138,000,or 67.1% of sales, in 1995, a decrease of 1.3% of sales for the
quarter and 1.7% for the nine month period. Amortization for the three and nine
months ended September 30, 1996, was $417,000 and $896,000, respectively,
compared to $356,000 and $930,000 in 1995, resulting in an increase of
approximately .5% of sales for both periods. Royalties for the three and nine
months ended September 30, 1996, were $231,000 and $496,000 or 4% of sales for
both periods, respectively, as compared to $160,000 and $470,000 or 2.9% and
3.5%, respectively, of sales for the same periods of 1995. The improvement in
gross profit margin resulted from higher margins on mini-book sales and sales of
first time titles plus an easing of paper prices partially offset by small
increases in amortization and royalties as a percent of sales.

Selling, general and administrative expenses for the three and nine month
periods ended September 30, 1996, increased to $997,000 and $3,187,000,
respectively, as compared to $955,000 and $3,042,000 for the corresponding
periods of 1995. This represents increases of $42,000 and $145,000 for the three
and nine month periods, respectively. These expenses comprised of personnel,
selling and administrative are generally fixed and do not fluctuate with sales.
Personnel expenses were $461,000 and $1,524,000, respectively, for the three
month and nine month periods ended September 30, 1996, as compared to $503,000
and


                                       5
<PAGE>   8
$1,601,000 for the corresponding periods of 1995. Declines of $42,000 and
$77,000 for the three and nine month periods were primarily attributable to the
Company's measures taken in 1995 to reduce employees and related personnel
expense partially offset by salary increases in the first quarter of 1996.
Selling expenses were $195,000 and $564,000, respectively, for the three and
nine month periods ended September 30, 1996, compared to $188,000 and $619,000
for the same periods in 1995. Selling expenses were up $7,000 for the quarter
and down $55,000 for the nine months ended September 30, 1996. The nine month
reduction as compared to the prior year included decreases in delivery, sample
expenses, and U.K. office expense partially offset by increases in travel and
show expenses. Administrative expenses were $341,000 and $1,099,000,
respectively, for the three month and nine month periods ended September 30,
1996, compared to $264,000 and $822,000 for the same periods of 1995.
Administrative expenses were up $77,000 and $277,000, respectively, for the
three and nine month periods. The increase for the three and nine months is
primarily related to increases in acquisition expenses, office expenses, other
taxes and directors' expenses. The Company anticipates increased administrative
expenses in the fourth quarter of 1996 because of severance and legal costs.

LIQUIDITY AND CAPITAL RESOURCES

Cash and cash equivalents were $1,063,000 at September 30, 1996, as compared to
$915,000 at December 31, 1995. Net cash provided by operating activities was
$1,455,000 as compared to $716,000 for the corresponding period of the previous
year. The $739,000 increase was primarily attributable to a decrease of
$1,321,000 in accounts receivable partially offset by a decrease of $321,000 in
accounts payable and an increase in prepaid expenses of $359,000 and a net loss
of $196,000 resulting from decreased sales for the nine month period and
improved collections of the outstanding receivables. Net cash used in investing
activities amounted to $1,307,000 as compared to $1,362,000 during the same
period in 1995. Working capital at September 30, 1996 was $4,556,000 as compared
to $4,654,000 at December 31, 1995.

Under a letter agreement, the Company has an obligation to fund the Hunt Group
$400,000 in 1996. This obligation is being paid at the rate of $33,333 per
month.

In June 1996 the Company renewed its $2,000,000 letter of credit line with City
National Bank which expires on May 31, 1997. The letter of credit facility is
available only for the issuance of letters of credit and as of September 30,
1996, the Company had $1,682,000 available under this letter of credit facility.

As of November 1, 1996, the Company did not have any commitments for any
material capital expenditures for 1996 or beyond. Management believes that the
existing levels of funds, combined with the Company's ability to generate cash,
are adequate to finance current and expected levels of activity as well as
anticipated capital expenditures of the Company for at least the next twelve
months.


                                       6
<PAGE>   9
PART II - OTHER INFORMATION

Item 4.  Submission of Matters to a Vote of Security-Holders

      The Annual Meeting of Shareholders of Intervisual Books, Inc. was held
on August 21, 1996, and several matters were put to a vote.  The issues and
the results are as follows:

(1)   To elect six directors to serve until the next annual meeting of
      shareholders and until their successors are chosen.

                                                               Abstentions
                   Votes Cast For    Votes Cast Against    Withhold Authority
                   --------------    ------------------    ------------------
Waldo H. Hunt
By Proxy              3,769,921               0                 225,000
In Person                     0               0                       0
Total                 3,769,921               0                 225,000
                      
Charles E. Gates      
By Proxy              3,769,921               0                 225,000
In Person                     0               0                       0
Total                 3,769,921               0                 225,000
                      
Elgin Davis           
By Proxy              3,769,421               0                 225,000
In Person                     0               0                       0
Total                 3,769,421               0                 225,000
                      
Gordon Hearne         
By Proxy              3,769,421               0                 225,000
In Person                     0               0                       0
Total                 3,769,421               0                 225,000
                      
John J. McNaughton    
By Proxy              3,769,921               0                 225,000
In Person                     0               0                       0
Total                 3,769,921               0                 225,000
                      
Peter Seymour         
By Proxy              3,769,921               0                 225,000
In Person                     0               0                       0
Total                 3,769,921               0                 225,000
                  
(2)   To ratify the selection of BDO Seidman, LLP, as independent auditors of
      the Company for 1996.

By Proxy              3,995,671           4,470                   1,710
In Person                     0               0                       0
Total                 3,995,671           4,470                   1,710


                                       7
<PAGE>   10
Item 5.  Other Information

      On November 1, 1996, the Company announced the resignation of Charles
E. Gates, the Company's President and Chief Executive Officer, effective
November 15, 1996.  Mr. Gates has agreed to provide consulting services to
the Company for a period of approximately five (5) years.  Mr. Hunt, the
Company's Chairman of the Board, will temporarily assume the responsibilities
of President and Chief Executive Officer until a successor for Mr. Gates is
appointed.

Item 6.  Exhibits and Reports on Form 8-K

(a)   Exhibits required by Item 601 of Regulation S-K
      11.   Computation of Earnings Per Common Share Schedule
      27.   Financial Data Schedule

(b)   Reports on Form 8-K
      None


Pursuant to the requirements of the Securities Exchange Act of 1934, the
Registrant has duly caused this report to be signed on its behalf by the
undersigned thereunto duly authorized.

                             INTERVISUAL BOOKS, INC.



                              BY:        /s/  Charles E. Gates
                                 -----------------------------------------------
                                 Charles E. Gates
                                 President, Chief Executive Officer
                                    and Chief Financial Officer




                              BY:        /s/  Gail A. Thornhill
                                 -----------------------------------------------
                                 Gail A. Thornhill
                                 Controller and Chief Accounting Officer



Date:  November 13, 1996
     -------------------


                                       8