<PAGE>

                                                                    EXHIBIT 10.4

                            HAWTHORNE SAVINGS, F.S.B.

                           DEFERRED COMPENSATION PLAN

                   Amended and Restated Effective May 1, 2003

<PAGE>

                            HAWTHORNE SAVINGS, F.S.B.

                           DEFERRED COMPENSATION PLAN

                                    ARTICLE I

                                     PURPOSE

         1.1      Establishment of the Plan. Hawthorne Savings, F.S.B. (the
"Company") hereby amends and restates the Hawthorne Savings, F.S.B. Deferred
Compensation Plan (the "Plan") effective May 1, 2003. The Plan was established
by the Company effective October 1, 2000, and previously amended on May 1, 2002.

         1.2      Purpose of the Plan. The purpose of the Plan is to permit a
select group of management or highly compensated employees of the Company to
defer payment of a portion of their current compensation, and to permit members
of the Company's board of directors (the "Board") to defer receipt of director's
fees. The Plan is unfunded and is intended to be exempt from the participation,
vesting, funding, and fiduciary requirements set forth in Title I of the
Employee Retirement Income Security Act of 1974, as amended ("ERISA").

                                   ARTICLE II

                                   DEFINITIONS

         2.1      Definitions. The following definitions are in addition to any
other definitions set forth elsewhere in the Plan. Whenever used in the Plan,
capitalized terms shall have the meanings set forth below unless otherwise
required by the context in which they are used:

                  (a)      "Cash Incentive Compensation" means all forms of cash
compensation earned by an Officer of the Company, other than base salary and
sign-on bonuses.

                  (b)      "Code" means the Internal Revenue Code of 1986, as
amended.

                  (c)      "Compensation" means, with respect to an Officer, his
or her base salary and Cash Incentive Compensation. "Compensation" means, with
respect to a Non-Employee Director, any compensation, whether regular retainer,
Board meeting fees, committee meeting fees, or otherwise earned by a
Non-Employee Director for services rendered in such capacity.

                  (d)      "Deferral Account" means the account established
pursuant to Article VI to reflect a Participant's deferred compensation and
adjustments thereto.

                  (e)      "Non-Employee Director" means any member of the
Company's Board of Directors who is not an employee of the Company.

                  (f)      "Officer" means any employee of the Company who is an
officer having the title of senior vice-president or a more senior title.

<PAGE>

                  (g)      "Participant" means any Officer or Non-Employee
Director who meets the eligibility requirements of the Plan, as set forth in
Article IV, and who elects to defer Compensation under the Plan; and includes,
where appropriate to the context, any former Officer or Non-Employee Director
who is entitled to benefits under this Plan.

                  (h)      "Plan Year" means the calendar year, except that the
initial Plan Year shall commence on October 1, 2000, and end on December 31,
2000.

                  (i)      "Unforeseeable Emergency" means a severe financial
hardship to a Participant resulting from a sudden and unexpected illness or
accident of the Participant or the Participant's dependent (as defined in
Section 152(a) of the Code), loss of the Participant's property due to casualty
or similar extraordinary and unforeseeable circumstance arising as a result of
events beyond the control of the Participant.

                  (j)      "Valuation Date" means the last day of each calendar
month and any other date designated as such by the Committee.

                                  ARTICLE III

                                 ADMINISTRATION

         3.1      Committee. The Plan shall be administered by the Compensation
Committee (the "Committee") of the Board, which shall consist of not fewer than
two members of the Board. The Board may from time to time appoint members of the
Committee in substitution for or in addition to members previously appointed and
may fill vacancies, however caused, in the Committee.

         3.2      Authority of the Committee. The Committee shall have absolute
authority in its discretion, to the maximum extent permissible by law, subject
to and not inconsistent with the express provisions of the Plan, to interpret
and administer the Plan and to make all determinations thereunder, including
determinations as to eligibility and as to the amount and timing of benefit
payments. In exercising its authority, subject only to applicable legal
limitations, the Committee shall be under no obligation or duty to treat
similarly situated Participants in the same manner, and any action taken by the
Committee with respect to one Participant shall in no way obligate the Committee
to take the same or similar action with respect to any other Participant. The
Committee may adopt such rules as it deems necessary or advisable in order to
carry out the purpose of the Plan. All questions of interpretation,
administration and application of the Plan shall be determined by a majority of
the members of the Committee then in office, except that the Committee may
authorize any one or more of its members, or any officer of the Company, to
execute and deliver documents on behalf of the Committee. Any interpretation or
determination made by the Committee shall be conclusive and binding upon any
person having or claiming any interest under the Plan and, in the event of
judicial review, shall be entitled to the maximum deference allowable by law.

         3.3      Authority of Board. Notwithstanding the foregoing, if the
Board does not appoint a Committee, the Board shall have all of the authority of
and fulfill all of the functions of the Committee hereunder. If the Board does
appoint a Committee, the Board may still reserve

                                       2

<PAGE>

certain authority and responsibility to itself with respect to the Plan. To the
extent necessary to be consistent with the provisions of this Section 3.3, any
reference in the Plan to a decision, determination or action of the Committee
shall be read and understood as referring to a decision, determination or action
of the Board.

         3.4      Liability of Board and Committee Members. Except as otherwise
required by law, no member of the Board or Committee shall be liable for
anything whatsoever in connection with the administration of the Plan other than
such member's own willful misconduct. In the performance of its functions with
respect to the Plan, the Board and Committee shall be entitled to rely upon
information and advice furnished by the Company's officers, accountants, legal
counsel and any other party the Board and Committee deems necessary, and no
member of the Board or Committee shall be liable for any action taken or not
taken in reliance upon any such advice.

                                   ARTICLE IV

                                   ELIGIBILITY

         4.1      Eligibility. Each Officer and Non-Employee Director shall
automatically be eligible to participate in the Plan.

                                   ARTICLE V

                                   DEFERRALS

         5.1      Deferrals. A Participant may elect to defer receipt of a
portion of his or her Compensation for a Plan Year. An election to defer
Compensation shall be made in writing prior to the first day of the Plan Year to
which it applies, except that with respect to the first Plan Year in which an
Officer or Non-Employee Director becomes eligible to participate in the Plan,
the election must be made no later than thirty (30) days after the date on which
such individual becomes eligible to participate in the Plan, and may only be
made with respect to Compensation earned for services performed after such
election.

                  All deferral elections shall be subject to the following
                  rules:

                  (a)      An Officer may elect to defer a whole percentage of
his or her base salary, and/or a whole percentage of his or her Cash Incentive
Compensation, which percentages need not be the same. Moreover, a deferral
election with respect to Cash Incentive Compensation may specify that the Cash
Incentive Compensation to be deferred for the Plan Year (i) shall not exceed a
stated dollar amount and/or (ii) shall not be less than a stated minimum dollar
amount (but not more than one hundred percent (100%) of the Cash Incentive
Compensation). Amounts deferred from an Officer's base salary shall reduce the
Officer's base salary in equal installments for each pay period during the Plan
Year (or portion thereof) to which the election applies. Amounts deferred from
an Officer's Cash Incentive Compensation shall reduce the Officer's Cash
Incentive Compensation for the Plan Year on the date such Cash Incentive
Compensation would otherwise be paid to such individual (which, in the case of
discretionary annual cash bonuses, is generally within forty five (45) days
following the end of the Plan Year).

                                       3

<PAGE>

                  (b)      A Non-Employee Director may elect to defer a whole
percentage of his or her Compensation.

                  (c)      An election shall be irrevocable with respect to all
Compensation payable for services performed by the Participant during the Plan
Year following the date on which the election is received by the Committee,
except that a Participant may terminate an election to defer Compensation if the
Committee determines that the termination is necessary as a result of an
Unforeseeable Emergency.

                  (d)      While an election to defer Compensation shall relate
solely to the Plan Year for which it is made, nonetheless, unless the
Participant files a new election form on a timely basis for a subsequent Plan
Year, a Participant's election form in effect for a Plan Year shall
automatically renew and become applicable to the next following Plan Year.

         5.2      Election of Form and Time of Payment.

                  (a)      At the time that a Participant makes a deferral
election pursuant to Section 5.1, the Participant shall also designate on the
Participant's deferral election form (i) the form in which the deferred amount
for such Plan Year (and earnings thereon) is to be paid by the Company, and (ii)
the time at which the deferred amount for such Plan Year (and earnings thereon)
is to be paid by the Company.

                  (b)      The available forms of payment under the Plan are:
(i) a single lump sum payment; (ii) annual installments over a period of five
(5) years; and (iii) annual installments over ten (10) years.

                  (c)      A Participant may designate (i) that payment is to be
made upon the Participant's attainment of a particular age; (ii) that payment is
to be made upon termination of the Participant's services as an Officer or
Non-Employee Director for any reason; or (iii) that payment is to be made upon
the Participant's attainment of a particular age, but not prior to termination
of the Participant's services. Subject to paragraphs (d) and (e), payment shall
be made (or, in the case of installments, commence) in accordance with the
Participant's election, as soon as practicable following the date designated by
the Participant.

                  (d)      Notwithstanding any election made by a Participant
pursuant to this Section 5.2, in the event that a Participant's service
terminates prior to the Participant's attainment of age sixty (60), the
Participant's entire Deferral Account balance shall be paid in a single lump sum
as soon as practicable following the Participant's termination of service.

                  (e)      Notwithstanding the foregoing, in the event that the
Committee determines that payment of an amount at the time elected by the
Participant would result in all or a portion thereof not being deductible for
federal income tax purposes, pursuant to Section 162(m) of the Code, the
Committee, in its sole discretion, may defer the payment (or a portion thereof)
until the earliest year in which the payment (or the portion) is fully
deductible.

                  (f)      Any election of a specified payment form or date
pursuant to this Section 5.2 shall be subject to any restrictions that the
Committee may, in its sole discretion,

                                       4

<PAGE>

choose to establish in order to limit the number of different payment forms or
dates that a Participant may have in effect at one time.

                  (g)      Except as otherwise provided in Sections 5.3 and 5.4,
all elections of forms and times of payment are irrevocable thirty (30) days
after they are made.

         5.3      Modification of Prior Election; Acceleration of Payment Date.

                  (a)      In the event of an Unforeseeable Emergency, the
Committee, in its sole discretion, may allow a Participant, to accelerate
payment hereunder and receive an amount which, under the original election form
filed by the Participant, would not be payable until a future date, but only to
the extent necessary to prevent severe financial hardship. Such action shall be
taken only if the Participant submits an application describing the
circumstances which the Participant believes justify the payment and the amount
necessary to prevent severe financial hardship, together with such supporting
evidence as the Committee may reasonably require. Payment may not be made under
this paragraph (a) to the extent that the Participant's hardship is or may be
relieved (i) through reimbursement or compensation by insurance or otherwise;
(ii) by liquidation of the Participant's assets, to the extent that this would
not in itself cause severe financial hardship; or (iii) by the termination of
the Participant's election to defer the payment of compensation hereunder.

                  (b)      A Participant may at any time change a prior election
and elect to accelerate payment hereunder by giving prior written notice thereof
to the Committee; provided, however, that any such election change shall result
in a reduction in the Participant's Deferral Account by ten percent (10%) of the
amount with respect to which the election change is being made, determined as of
the date of the election change.

         5.4      Modification of Prior Election: Deferral of Payment Date or
Change in Form of Payment. A Participant may at any time change a prior election
and elect to (i) delay the date on which payment is to be made (or, in the case
of installments, commence) or (ii) change the form in which payment is to be
made, by giving prior written notice thereof to the Committee; provided,
however, that any such election change shall not be effective unless it is made
no less than one year prior to the date on which payment would otherwise have
been made (or commenced), absent the election change.

         5.5      Restriction on Payment. Notwithstanding the above paragraphs,
any fixed payment date elected by a Participant as provided under Article V must
be no earlier than the January 1 of the third calendar year after the calendar
year in which the election is made or in which the Participant gives a written
notice of extension.

                                   ARTICLE VI

                                DEFERRAL ACCOUNTS

         6.1      Accounts. The Committee shall establish and maintain a
Deferral Account in the name of each Participant to record the deferred
compensation payable to the Participant hereunder, and shall, if applicable,
establish and maintain subaccounts for each deferral election of each
Participant that could cause a payment to be made at a different time or form
from other

                                       5

<PAGE>

payments of deferrals elected by the same Participant. A Participant's Deferral
Account shall be for bookkeeping purposes only and shall not be deemed to create
a fund or trust for the benefit of any Participant. A Participant's Deferral
Account shall be fully and immediately vested.

         6.2      Credits and Adjustments. Each Participant's Deferral Account
shall periodically be adjusted as follows:

                  (a)      Amounts deferred by a Participant pursuant to Article
V shall be credited to the Participant's Deferral Account as of the dates as of
which they are applied to reduce the Participant's applicable Compensation.

                  (b)      As of each Valuation Date, a Participant's Deferral
Account balance shall be credited with interest at a rate equal to the interest
rate on 10-year Treasury notes, as in effect on the last date of the calendar
year quarter immediately preceding the calendar year quarter in which occurs
such Valuation Date, plus two and one-half percent (2.5%).

                  (c)      As of the date of any distribution to a Participant,
a Participant's Deferral Account shall be reduced by the amount of the
distribution and, if applicable, by the amount of any penalty imposed, pursuant
to Section 5.3(b).

                  (d)      Payment of Deferred Amounts. A Participant's Deferral
Account shall be paid by the Company to the Participant in accordance with the
Participant's deferral elections, but subject to the provisions of Section 5.2.
If such payment is to be made in the form of a lump sum, it shall be in an
amount equal to the balance of the Deferral Account (or the subaccount) being
paid, determined as of the Valuation Date coincident with or immediately
preceding the payment date. If the payment is to be made in the form of
installments, the amount of each installment payment shall be equal to the
quotient determined by dividing (i) the value of the portion of the Deferral
Account (or the subaccount) to which the installment payment election applies,
determined as of the Valuation Date coincident with or immediately preceding the
date the payment is to be made, by (ii) the number of years over which the
installment payments are to be made, less the number of years in which prior
payments attributable to such installment payment election have been made.

                                  ARTICLE VII

                               GENERAL PROVISIONS

         7.1      Unfunded Obligation. Deferred amounts to be paid to
Participants pursuant to the Plan shall constitute unfunded obligations of the
Company. The Deferral Accounts provided by the Plan are for accounting purposes
only and shall not be funded by a trust or by any segregated assets.

         7.2      Informal Funding Vehicles to Facilitate Payment. The Company
may, but need not, arrange for the purchase of insurance contracts or other
assets and may, but need not establish a so-called "rabbi trust" or other
informal funding vehicle to facilitate the payment of benefits and to discharge
the liability of the Company under the Plan. The making of any such investments
and/or the creation or maintenance of memorandum accounts or a rabbi trust or
other informal funding vehicles shall not, however, be deemed to create a trust
or a fiduciary

                                       6

<PAGE>

relationship between the Committee or the Company and a Participant or otherwise
confer on any Participant or his or her creditors a vested or beneficial
interest in any assets of the Company whatsoever. Participants have no claim
against the Company for any changes in the value of any assets which may be
invested or reinvested by the Company with respect to the Plan.

         7.3      Status of Participants. Each Participant shall be a general
unsecured creditor of the Company with respect to amounts payable hereunder, the
Plan constituting a mere promise by the Company to make benefit payments in the
future.

         7.4      Beneficiary. For purposes hereof a "Beneficiary" shall mean
the person or persons to whom payments are to be paid pursuant to the terms of
the Plan in the event of a Participant's death. A Participant may designate a
Beneficiary on a form provided by the Committee, executed by the Participant,
and delivered to the Committee; provided, however, that if a married Participant
designates a Beneficiary other than not them the Participant's spouse, the
designation shall be valid only if consented to, in a notarized writing, by the
Participant's spouse. Subject to the foregoing, a Participant may change a
Beneficiary designation at any time. Subject to the property rights of any prior
spouse, if no Beneficiary is designated, if the designation is ineffective, or
if the Beneficiary dies before the balance of the Account is paid, the balance
shall be paid to the Participant's surviving spouse, or if there is no surviving
spouse, to the Participant's estate.

         7.5      Incapacity of Participant or Beneficiary. Every person
receiving or claiming benefits under the Plan shall be conclusively presumed to
be mentally competent and of age until the date on which the Committee receives
a written notice, in a form and manner acceptable to the Committee, that such
person is incompetent or a minor, for whom a guardian or other person legally
vested with the care of his person or estate has been appointed; provided,
however, that if the Committee finds that any person to whom a benefit is
payable under the Plan is unable to care for his or her affairs because of
incompetency, or because he or she is a minor, any payment due (unless a prior
claim therefor shall have been made by a duly appointed legal representative)
may be paid to the spouse, a child, a parent, a brother or sister, or to any
person or institution considered by the Committee to have incurred expense for
such person or otherwise entitled to payment. To the extent permitted by law,
any such payment so made shall be a complete discharge of liability therefor
under the Plan. If a guardian of the estate of any person receiving or claiming
benefits under the Plan is appointed by a court of competent jurisdiction,
benefit payments may be made to such guardian provided that proper proof of
appointment and continuing qualification is furnished in a form and manner
acceptable to the Committee. In the event a person claiming or receiving
benefits under the Plan is a minor, payment may be made to the custodian of an
account for such person under the Uniform Gifts to Minors Act. To the extent
permitted by law, any such payment so made shall be a complete discharge of any
liability therefor under the Plan.

         7.6      Nonassignment. The right of a Participant to the payment of
any amounts under the Plan may not be assigned, transferred, pledged or
encumbered nor shall such right or other interests be subject to attachment,
garnishment, execution or other legal process.

         7.7      No Right to Continued Employment. Nothing in the Plan shall be
construed to confer upon any Participant any right to continued employment with
the Company, nor shall the

                                       7

<PAGE>

Plan interfere in any way with the right of the Company to terminate the
employment of any Participant at any time without assigning any reason therefor.

         7.8      No Guarantee of Board Membership. No provision in this Plan
shall be deemed or construed to impair or affect in any manner whatsoever the
right of the Company and its shareholders to discontinue or terminate the
services of any member of the Board to the same extent as if this Plan were not
in existence and had not been established.

         7.9      Tax Withholding. Appropriate taxes shall be withheld from cash
payments made to Participants pursuant to the Plan. To the extent tax
withholding is payable in connection with the Participant's deferral of income,
rather than in connection with the payment of deferral amounts, such withholding
may be made from other wages and salary currently payable to the Participant,
or, as determined by the Committee, the amount of the deferral elected by the
Participant may be reduced in order to satisfy required tax withholding for
employment taxes and any other taxes.

         7.10     Claims Procedure.

                  (a)      If a claim for benefits is denied, the Committee
shall furnish to the claimant within ninety (90) days after its receipt of the
claim (or within one hundred eighty (180) days after such receipt if special
circumstances require an extension of time) a written notice which (i) specifies
the reasons for the denial; (ii) refers to the pertinent provisions of the Plan
on which the denial is based; (iii) describes any additional material or
information necessary for the perfection of the claim and explains why such
material or information is necessary; and (iv) explains the claims review
procedures.

                  (b)      Upon the written request of the claimant submitted
within sixty (60) days after his or her receipt of such written notice, the
Committee shall afford the claimant a full and fair review of the decision
denying the claim and, if so requested, permit the claimant to review any
documents which are pertinent to the claim, permit the claimant to submit issues
and comments in writing and afford the claimant an opportunity to meet with
appropriate representatives of the Committee as a part of the review procedure.
Within sixty (60) days after its receipt of a request for review (or within one
hundred twenty (120) days after such receipt if special circumstances, such as
the need to hold a hearing, require an extension of time) the Committee shall
notify the claimant in writing of its decision and the reasons for its decision
and shall refer the claimant to the provisions of the Plan which form the basis
for its decision.

         7.11     Amendment, Suspension and Termination of Plan. The Company may
from time to time amend, suspend or terminate the Plan, in whole or in part.
Without limiting the foregoing, any amendment, suspension, or termination of the
Plan may, without the Consent of any Participant, (i) affect a Participant's
ability to make future deferrals; and (ii) on a prospective basis, affect the
rate at which interest is credited to a Participant's existing Deferral Account
under Section 6.2. Notwithstanding the foregoing, no amendment, suspension or
termination of the Plan may, without the consent of the affected Participant,
impair the right of a Participant to receive his or her existing Deferral
Account balance in accordance with the terms of the Plan prior to the effective
date of such amendment, suspension or termination; provided, however,

                                       8

<PAGE>

that upon a termination of the Plan, the Company may provide for the accelerated
distribution of Deferral Account balances.

         7.12     Applicable Law. To the extent not preempted by federal law,
the Plan shall be construed and governed in accordance with the internal laws of
the State of California.

         IN WITNESS WHEREOF, Hawthorne Savings, F.S.B. has caused this Plan to
be executed by its duly authorized officer.

                                               HAWTHORNE SAVINGS, F.S.B.:

                                               By: /s/ EILEEN LYON
                                                  ------------------------------
                                                  Eileen Lyon
                                                  Senior Vice President
                                             Date: April 22, 2003
                                                  ------------------------------
                                       9

<PAGE>

                                TABLE OF CONTENTS

<TABLE>
<CAPTION>
                                                                                           PAGE
                                                                                           ----
<S>                                                                                        <C>
ARTICLE I           PURPOSE...........................................................       1

         1.1    Establishment of the Plan.............................................       1

         1.2    Purpose of the Plan...................................................       1

ARTICLE II          DEFINITIONS.......................................................       1

         2.1    Definitions...........................................................       1

ARTICLE III         ADMINISTRATION....................................................       2

         3.1    Committee.............................................................       2

         3.2    Authority of the Committee............................................       2

         3.3    Authority of Board....................................................       2

         3.4    Liability of Board and Committee Members..............................       3

ARTICLE IV          ELIGIBILITY.......................................................       3

         4.1    Eligibility...........................................................       3

ARTICLE V           DEFERRALS.........................................................       3

         5.1    Deferrals.............................................................       3

         5.2    Election of Form and Time of Payment..................................       4

         5.3    Modification of Prior Election; Acceleration of Payment Date..........       5

         5.4    Modification of Prior Election:  Deferral of Payment Date or Change in
                Form of Payment.......................................................       5

         5.5    Restriction on Payment................................................       5

ARTICLE VI          DEFERRAL ACCOUNTS.................................................       5

         6.1    Accounts..............................................................       5

         6.2    Credits and Adjustments...............................................       6

ARTICLE VII         GENERAL PROVISIONS................................................       6

         7.1    Unfunded Obligation...................................................       6

         7.2    Informal Funding Vehicles to Facilitate Payment.......................       6

         7.3    Status of Participants................................................       7

         7.4    Beneficiary...........................................................       7

         7.5    Incapacity of Participant or Beneficiary..............................       7

         7.6    Nonassignment.........................................................       7

         7.7    No Right to Continued Employment......................................       7
</TABLE>

                                      -i-

<PAGE>

                               TABLE OF CONTENTS
                                  (continued)

<TABLE>
<CAPTION>
                                                                                  PAGE
                                                                                  ----
<S>                                                                               <C>
7.8    No Guarantee of Board Membership......................................       8

7.9    Tax Withholding.......................................................       8

7.10   Claims Procedure......................................................       8

7.11   Amendment, Suspension and Termination of Plan.........................       8

7.12   Applicable Law........................................................       9
</TABLE>

                                      -ii-

<PAGE>

                                 AMENDMENT NO. 1
                                     TO THE
                            HAWTHORNE SAVINGS, F.S.B.
                           DEFERRED COMPENSATION PLAN
                  (AMENDED AND RESTATED EFFECTIVE MAY 1, 2003)

         Pursuant to Section 7.11 of the Hawthorne Savings, F.S.B. Deferred
Compensation Plan, as amended and restated effective May 1, 2003 (the "Plan"),
the Plan is hereby amended, effective August 18, 2003, in the following respects
only:

         A new Subsection 2.1(k) is added to the Plan to read as follows:

                  (k)      "Loan Officer" means an employee of the Company who
         is designated as a loan officer and holds the title of vice president.

         Section 4.1 of the Plan is amended in its entirety to read as follows:

         4.1      Eligibility. Each Officer and Non-Employee Director
         automatically shall be eligible to participate in the Plan. Each Loan
         Officer named by the Committee or satisfying requirements or criteria
         established from time to time by the Committee also shall be eligible
         to participate in the Plan.

         IN WITNESS WHEREOF, Hawthorne Savings, F.S.B. has caused this amendment
to be executed by its duly authorized officer on this 18th day of August,
2003.

                                                 HAWTHORNE SAVINGS, F.S.B.

                                                    By: /s/ EILEEN LYON
                                                       -------------------------

                                                 Title: Senior Vice President
                                                       -------------------------