<PAGE>   1
                                                                   EXHIBIT 10.41

                       SCPIE HOLDINGS, INC. & SUBSIDIARIES
                    CONSOLIDATED FEDERAL INCOME TAX LIABILITY
                              ALLOCATION AGREEMENT

        This Agreement, effective as of the first day of the consolidated return
year beginning January 1, 1996, is made by and among SCPIE Holdings, Inc.
("Parent") and Southern California Physicians Insurance Exchange, SCPIE
Management Company, SCPIE Indemnity Company, American Healthcare Indemnity, FG
Casualty Company, SCPIE Insurance Services, Inc., and SCPIE Management Services
("Subsidiaries").

        WHEREAS, the parties hereto (hereinafter sometimes referred to as
"Members"; or in the singular "Member" or "party") are part of an affiliated
group ("Affiliated Group") as defined by Section 1504(a) of the Internal Revenue
Code of 1986 ("IRC"), as amended;

        WHEREAS, such Affiliated Group has filed a consolidated federal income
tax return since their affiliation in accordance with IRC Section 1501 and is
required to continue filing consolidated income tax returns for years subsequent
to such year of first consolidated filing; and

        WHEREAS, it is the intent and desire of the parties hereto that a method
be established for allocating the consolidated "federal income tax liability"
(as determined under Regulations Section 1.1502-2) of the Affiliated Group among
its Members (as required by IRC Section 1552(a)); for reimbursing the Parent for
payment of such tax liability; for compensating any Member for use of its "net
operating loss," "net capital loss," or "tax credits" in arriving at such tax
liability; and to provide for the allocation and payment of any refund arising
from a carryback of net operating losses, net capital losses, or tax credits
from subsequent taxable years.

        NOW THEREFORE, in consideration of the mutual covenants and promises
contained herein, the parties hereto agree as follows:

        1.     A U.S. consolidated federal income tax return shall be filed by
               the Parent for the taxable year ended December 31, 1996, and for
               each subsequent taxable year in respect of which this Agreement
               is in effect and for which the Affiliated Group is required or
               permitted to file a consolidated federal income tax return.

        2.     The Parent and Subsidiaries have hereby elected to allocate their
               federal tax liability during consolidated tax return years
               encompassed by this Agreement pursuant to Regulation Section
               1.1552-1(a)(2)(i) in conjunction with the election pursuant to
               Regulation Section 1.1502-33(d)(3). The fixed percentage to be
               used under Regulation Section 1.1502-33(d)(3)(i) shall be 100%.

        3.     The provisions of this Agreement shall be administered by the
               Chief Financial Officer of Parent.

                                      -1-
<PAGE>   2

        4.     Each Member shall reimburse the Parent for its allocated
               consolidated federal income tax liability under this Agreement.
               Each Member benefiting from net operating losses, net capital
               losses, and tax credits shall pay to the Parent its added tax
               assessment determined under paragraph 2 of this Agreement. The
               Parent shall pay to each Member with a net operating loss, net
               capital loss, or tax credit(s) during the taxable year its
               allocable share of the total of the additional amounts due from
               other Members pursuant to paragraph 2 of this Agreement.
               Payments for these allocable shares are to be made no later than
               thirty (30) days after the date of filing of the consolidated
               federal income tax return for such taxable year. No payment
               shall be made a Member for the benefit of a net operating loss,
               net capital loss, or tax credit unless such net operating loss,
               net capital loss, or tax credit is availed of in reducing the
               consolidated federal income tax liability.

        5.     Notwithstanding the provisions of paragraph 2 and 4, it is
               agreed as follows:

               Alternative Minimum Tax ("AMT") due on the federal consolidated
               tax return is, generally, to be allocated in an equitable and
               consistent manner pursuant to paragraph 2, based on the separate
               company's contribution to the AMT adjustments and preference
               items which result in the consolidated AMT. Tax benefits from
               utilization of AMT net operating losses (including, but not
               limited to, amounts for negative adjusted current earnings
               adjustments) and AMT credits are to be similarly allocated.

        6.     The Chief Financial Officer of Parent shall have the right to
               assess Members their share of estimated tax payments based upon
               the amount of tax estimated to be due under paragraph 2 and
               currently payable to taxing authorities. Payment to the Chief
               Financial Officer of Parent shall be made thirty (30) days after
               such assessment. The assessment will not be made more than two
               (2) days prior to the required federal income tax payment dates.
               Such Member will receive credit for such prepayments in the year
               end computation under paragraph 2 and 4 of this Agreement.

        7.     If part or all of an unused consolidated net operating loss, net
               capital loss or tax credit is allocated to a Member of the
               Affiliated Group pursuant to Regulations Section 1.1502-79, and
               it is carried back or forward to a year in which such Member
               filed a separate income tax return or a consolidated federal
               income tax return with another affiliated group, any refund or
               reduction in tax liability arising from the carryback or
               carryover shall be retained by such Member. (If such refund or
               reduction goes to some entity other than the Member, then such
               entity shall pay over such amount to the Member).
               Notwithstanding the above, the Parent shall determine whether an
               election shall be made to forego carryback of any consolidated
               net operating loss or tax credit arising in a consolidated
               return year (including any portion allocated to a Member under
               Regulations Section 1.1502-79) in accordance with Section
               172(b)(3) or other IRC provisions which relate to tax credit
               utilization.

                                      -2-
<PAGE>   3

        8.      If the consolidated federal income tax liability is adjusted for
                any taxable period encompassed by this Agreement, whether by
                means of amended return, claim for refund, or tax audit by the
                Internal Revenue Service, the liability of each Member shall be
                recomputed under paragraph 2 of this Agreement to give effect to
                such adjustments. In the case of a refund, the Parent shall make
                payment to each Member for its share of the refund, determined
                in the same manner as in paragraph 4 of this Agreement, within
                thirty (30) days after the refund is received by the Parent, and
                in the case of an increase in tax liability, each Member shall
                pay to the Parent its allocable share of such increased tax
                liability within thirty (30) days after receiving notice of such
                liability from the Parent. If any interest is to be paid or
                received as a result of a consolidated federal income tax
                deficiency or refund, such interest shall be allocated to the
                Members in the ratio each Member's change in consolidated
                federal income tax liability bears to the total change in tax
                liability. Any penalty or interest shall be allocated upon such
                basis as the Chief Financial Officer of Parent deems just and
                proper in view of all applicable circumstances.

        9.      Notwithstanding the above, in no instance shall Southern
                California Physicians Insurance Exchange, SCPIE Indemnity
                Company, American Healthcare Indemnity, and FG Casualty Company
                (hereinafter referred to as "Insurance Company Members"; or in
                the singular "Insurance Company Member") assume more tax
                liability, make greater tax payment, or receive a lesser refund
                than the Insurance Company Member would have assumed, made or
                received as a taxpayer filing a separate income tax return. In
                this regard, Parent hereby agrees to indemnify, defend or hold
                the Insurance Company Members harmless from and against any levy
                on the assets of the Insurance Company Members by the Internal
                Revenue Service which results from the Insurance Company Members
                having filed consolidated federal income tax returns with the
                Affiliated Group pursuant to this Agreement. Such
                indemnification shall be limited to the lesser of: (i) the
                amount of taxes which the Insurance Company Member would have
                been liable for if the Insurance Company Member filed a separate
                income tax return or (ii) Insurance Company Member's allocable
                share of consolidated federal income taxes paid (including
                interest and penalties) as a Member of the Affiliated Group and
                pursuant to this Agreement.

                The Insurance Company Members agree to promptly provide Parent
                written notice of any claim made on the obligation indemnified
                against hereunder. Written notice to the Parent will be deemed
                promptly made if such notification is made within five (5) days
                following receipt, by an Insurance Company Member, of any notice
                of claim by the Internal Revenue Service. Indemnity set forth
                herein shall extend to each taxable year in which the Insurance
                Company Member 's tax liability (or refund) is determined under
                the provisions of this Agreement.

                                      -3-
<PAGE>   4

        10.    This Agreement shall apply to the taxable years specified in the
               preamble of this Agreement, and all subsequent taxable years,
               unless the Members agree in writing to terminate the Agreement.
               Notwithstanding such termination, this Agreement shall continue
               in effect with respect to any payment or refunds due for all
               taxable periods prior to termination.

        11.    This Agreement shall be not assignable by any Member without the
               prior written consent of the other Members.

        12.    All materials including, but not limited to, returns, supporting
               schedules, work papers, correspondence, and other documents
               relating to the consolidated federal income tax returns filed for
               a taxable year during which this Agreement was in effect shall be
               made available to any Member to the Agreement during the regular
               business hours for a minimum period equal to applicable record
               retention requirements of the Internal Revenue Service.

        13.    The Members shall apply the principles set forth herein to State
               and Local tax liabilities to the extent the Members are included
               in a consolidated, combined, or unitary state tax filing with any
               Member(s).

        14.    Any dispute or difference between the parties with respect to
               the operation or interpretation of this Agreement shall be first
               mediated by the Chief Executive Officer of the Parent ("CEO")
               and if mediation by the CEO is unsuccessful (or if mediation by
               the CEO would represent a conflict of interests), then to
               binding arbitration. The arbitrators must all be certified
               public accountants ("CPAs") or attorneys. Each Member shall
               elect an arbitrator. The losing party will bear all costs of
               arbitration including all fees for attorneys and accountants.

        15.    Any Member corporation which leaves the consolidated group shall
               be bound by this Agreement.

        16.    The Members hereto specifically recognize that from time to time
               other companies may become Members of the Affiliated Group and
               hereby agree that such new Members may become parties to this
               Agreement by executing the master copy of this Agreement which
               shall be maintained by the Parent's Corporate Secretary. It will
               not be necessary for all the other Members to resign the
               Agreement but the new Member may simply sign the existing
               Agreement and it will be effective as if the old Members had
               resigned.

        17.    Any alteration, modification, addition, deletion, or other change
               in the consolidated income tax return provision of the Code or
               the regulations thereunder shall automatically be applicable to
               this Agreement.

        18.    Failure of one or more parties hereto to qualify by meeting the
               definition of Member of the "Affiliated Group" shall not operate
               to terminate this Agreement


                                      -4-
<PAGE>   5

                with respect to the other parties as long as two or more parties
                hereto continue to qualify.

        19.     During any consolidated tax year to which this Agreement, any
                Member whose affiliation with the Affiliated Group spans for a
                period of less than 12 months ("short-year") shall be bound by
                this Agreement at all times in which such Member meets the
                definition of Member of the Affiliated Group. Such short-year
                Member shall be allocated its share of the consolidated federal
                income tax liability in an equitable and consistent manner
                pursuant to paragraph 2 based on the Member's contribution to
                the consolidated federal income tax liability.

        20.     This Agreement shall bind and inure to the respective successors
                and assigns of the parties hereto; but no assignment shall
                relieve any party's obligations hereunder without the written
                consent of the other parties.

        21.     If any provision of this Agreement, as applied to any party
                hereto or to any circumstances involving the subject matter
                hereof, shall be found by a court of competent jurisdiction to
                be void, invalid, or unenforceable, then, notwithstanding such
                provision, all other provisions of this Agreement shall remain
                in effect and binding on the parties hereto to the fullest
                extent permitted by law.

        22.     This Agreement contains the entire understanding of the parties
                hereto relating to the subject matter hereof and supersedes all
                prior and collateral agreements, understandings, statements, and
                negotiations of the parties. Each party hereto acknowledges that
                no representations, inducements, promises, or other agreements,
                oral or written, with reference to the subject matter hereof
                have been made other than as expressly set forth herein.

        IN WITNESS WHEREOF, the parties hereto have caused their names to be
        subscribed and executed by their respective authorized officers on the
        dates indicated, effective as of the date first written above.

SCPIE Holdings, Inc.


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title

Southern California Physicians Insurance Exchange


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title


                                      -5-
<PAGE>   6
SCPIE Management Company


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title


SCPIE Indemnity Company


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title

American Healthcare Indemnity


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title


FG Casualty Company


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title


SCPIE Insurance Services, Inc.


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title

SCPIE Management Services


By: /s/ PATRICK T. LO                    Date  5/9/97
   ------------------------------------       --------------------------------
    Name/Title

                                      -6-