<PAGE>


                                                                   Exhibit 10.4

                   AMENDED AND RESTATED SPLIT DOLLAR AGREEMENT


                            ALLMERICA LIFE INSURANCE
                              POLICY NO. V062414100


         AGREEMENT made as of this 18th day of September, 2002, by and between
Dennis Adomaitis (the "Employee"), and The J. Jill Group, Inc., a Delaware
corporation (the "Employer").

         WHEREAS, the Employee wishes to establish a life insurance program for
the benefit and protection of his family under Policy No. V062414100 (the
"Policy") issued by Allmerica Financial Life Insurance and Annuity Company, of
Worcester, Massachusetts (the "Insurer"); and

         WHEREAS, the Employer wishes to help the Employee provide such
insurance for the benefit and protection of his family by the payment of the
premiums due on the Policy in accordance with Article 2 hereof; and

         WHEREAS, the Employee will be the sole owner of the Policy, and will
assign the Policy to the Employer for the purpose of providing security for the
repayment of the amounts which the Employer will contribute toward payment of
the premium due or to become due on the Policy pursuant to an Assignment of Life
Insurance Policy as Collateral of even date in the form attached as SCHEDULE 1
hereto (the "Collateral Assignment"); and

         WHEREAS, it is the desire of the parties to define the extent of the
Employer's interest in the cash surrender value and death proceeds of the
Policy;

         NOW THEREFORE, in consideration of the mutual promises contained
herein, it is agreed between the parties hereto as follows:

ARTICLE 1:  OWNERSHIP OF POLICY

         The Policy is the exclusive property of the Employee, who may exercise
all rights of ownership with respect to his interest therein, subject to the
rights and security interest of the Employer as expressed in this Agreement and
the Collateral Assignment and to any death benefit which may become due to the
Employer.

ARTICLE 2:  PAYMENT OF PREMIUMS

         A. For the first seven (7) years the Policy is in force, or until this
Agreement is earlier terminated as provided in Article 8, the Employer shall be
responsible for the payment of (i) the scheduled premium, or (ii) such lesser
amount as the Insurer advises is consistent with the insurance features of the
Policy. In addition, after the expiration of said seven-year period and while
this Agreement is still in effect, the Employer may make such additional
payments as the Insurer advises is consistent with maintaining the intended
value of the Policy but only if such


<PAGE>

greater amount or additional payments are approved by the Compensation Committee
of the Employer's Board of Directors.

         B. The Employee understands that he will recognize taxable income in
each year with respect to the life insurance protection afforded to the Employee
in accordance with Internal Revenue Service rulings and regulations.

         C. The Employer shall, before the end of any grace period provided in
the Policy for each premium payment, remit the annual premiums as stated in the
Policy until this Agreement is terminated as provided in Article 8. If
requested, the Employer shall give proof to the Employee of the timely payment
of each premium.

ARTICLE 3:  COLLATERAL ASSIGNMENT

         To secure the repayment to the Employer of an amount equal to the
aggregate amount of its premium payments under the Policy to the extent provided
in Article 6, the Employee has contemporaneously with the execution of this
Agreement assigned the Policy to the Employer as collateral, by means of the
form of Collateral Assignment attached to this Agreement as Schedule 1. The
Collateral Assignment shall not be altered, terminated or amended by Employee
without the express written consent of the Employer. The parties agree to take
all action necessary to cause such assignment to conform to the provisions of
this Agreement.

ARTICLE 4:  RIGHTS IN POLICY

         A. The Employer shall have no right to borrow against the Policy. The
Employer shall have the right to withdraw all or part of the cash surrender
value as defined in the Policy at any time and from time to time, up to the
amount of the Premium Reimbursement (as defined in Article 6, Section A);
PROVIDED, HOWEVER, that upon the occurrence of a Change in Control (as defined
in Article 6, Section E), the Employer's right to withdraw the cash surrender
value shall cease.

         B. The Employee, in recognition of the defined contribution provisions
of this Agreement and the variable nature of the death benefits, shall have the
right to allocate the aggregate account value to particular investment vehicles,
subject to a right of the Employer to disapprove a particular investment vehicle
which it deems inappropriate.

         C. The Employee shall have the right to exchange the Policy for such
other policies and/or insurers that he deems appropriate based upon the
investment performance or financial condition of the Insurer, subject to the
approval of the Employer, which approval shall not unreasonably be denied.
Action by the Employee or the Employer to change the Policy and/or insurer
pursuant to this paragraph shall not otherwise alter the rights and
responsibilities of the Employer and the Employee as set forth in this
Agreement.

         D. The Employer shall have no responsibility for a shortfall in the
projected total return on the paid-in premiums available to provide the death
benefit.


                                      -2-

<PAGE>

         E. Except as authorized by Section A of the Article 4, the Employer
shall not take any action that might endanger the interest of the Employee in
the Policy. The Employee shall not take any action that might endanger the
interest of the Employer in the Policy.

ARTICLE 5:  EMPLOYEE'S RIGHTS IN POLICY

         The Employee retains all other rights in the Policy not specifically
assigned to the Employer including, but not limited to, the following rights:

         A. The right to surrender the Policy as set forth in Paragraph D of the
Collateral Assignment.

         B. The right to change the beneficiary of the Policy, to the extent of
his interest.

         C. The right to select optional methods of settlement with regard to
the death benefit provided in PART TWO of Article 7.

         D. All other rights contained in the Policy, to the extent the exercise
of such rights does not adversely affect the Employer's interest in the Policy.

ARTICLE 6:  PAYMENT OF CASH SURRENDER VALUE

         A. Except as set forth in this Article 6, Section B, in the event this
Agreement is terminated pursuant to Section A or B of Article 8, the Employer
shall have the unqualified right to receive from the Insurer an amount which is
equal to the lesser of (a) the then cash surrender value as defined in the
Policy, or (b) the aggregate amount of premium payments with respect to the
Policy for which the Employer was responsible pursuant to Article 2, Section A
(the "Premium Reimbursement"), less any outstanding withdrawal by the Employer
of the cash surrender value of the Policy pursuant to Article 4, Section A. This
amount shall be established in a written statement to the Insurer by the
Employer, and the Insurer shall have the right, without liability to the
Employee or his beneficiary or beneficiaries of the Policy, to rely exclusively
upon such statement. The Employer shall, upon receiving such sum, release the
Collateral Assignment of the Policy.

         B. In the event that this Agreement is terminated pursuant to Section B
of Article 8 after the occurrence of a "Terminating Event" (as defined in
Section C of this Article 6), the Employer shall not be entitled to receive the
Premium Reimbursement pursuant to Section A of Article 6.

         C. For purposes of this Article 6, a "Terminating Event" shall mean any
of the following if it occurs within two years of a "Change in Control" (as
defined in Section E of this Article 6):

               (i) termination by the Employer of the Employee's employment with
the Employer for any reason other than (a) the Employee's death or disability,
or (b) for "Cause" (as such term is defined in Section D of this Article 6), or

               (ii) Employee's resignation as an employee of the Employer, other
than for reasons of disability, following a significant reduction in the nature
or scope of the Employee's duties, responsibilities, authority and


                                      -3-

<PAGE>

powers from the duties, responsibilities, authority and powers exercised by the
Employee immediately prior to the Change in Control or a reduction in the
Employee's annual base salary as in effect on the date of the Change in Control,
except for across-the-board salary reductions similarly affecting all management
personnel of the Employer (or the surviving entity, in the case of a merger or
acquisition in which the Employer is not the surviving entity).

         D.  For purposes of Section C of this Article 6, "Cause" shall mean:

               (i) deliberate dishonesty with respect to the Employer or any
subsidiary or affiliate thereof;

               (ii) conviction of a crime involving moral turpitude; or

               (iii) gross and willful failure to perform a substantial portion
of the Employee's duties and responsibilities as an officer of the Employer,
which failure continues for more than thirty days after written notice given to
the Employee pursuant to a two-thirds vote of all of the members of the Board of
Directors of the Employer then in office, such vote to set forth in reasonable
detail the nature of such failure.

         E. For the purposes of this Article 6, "Change in Control" shall mean
the occurrence of any one or more of the following events:

               (i) if there is a merger or consolidation of the Employer with
any other entity and the voting securities of the Employer outstanding
immediately prior to such merger or consolidation do not continue to represent
(either by remaining outstanding or by being converted into voting securities of
the surviving entity) more than fifty percent (50%) of the combined voting power
of the voting securities of the Employer or such surviving entity immediately
after such merger or consolidation, or

               (ii) when any person or entity or group of persons or entities
either related or acting in concert becomes the "beneficial owner" (as defined
in Rule 13d-3 under the Securities Exchange At of 1934, as amended) of
securities of the Employer representing more than fifty percent (50%) of the
total number of votes that may be cast for the election of directors of the
Employer (any such person or entity or group of persons or entities being
referred to, collectively in the case of any such group, as an "Acquiring
Person"), or

               (iii) if the Employer sells all or substantially all of its
assets to another entity, other than in a transaction in which the voting
securities of the Employer outstanding immediately prior to such transaction
continue to represent (either by remaining outstanding or by being converted
into voting securities of the surviving entity) more than fifty percent (50%) of
the combined voting power of the voting securities of the Employer or such
surviving entity immediately after such transaction, or

               (iv) during any period of two consecutive years (not including
any period prior to the execution of this Agreement), individuals who are
Continuing Directors (as hereinafter defined) cease for any reason to constitute
at least a majority of the Board of Directors of the Employer. For this purpose,
a "Continuing Director" shall mean (a) an individual who was a director of the
Employer at the beginning of such period or (b) any new director (other than a


                                      -4-

<PAGE>

director designated by an Acquiring Person) whose election by the Board or
nomination for election by the Board, a committee thereof or the Employer's
stockholders was approved by a vote of a majority of the directors then still in
office who either were directors at the beginning of such period or whose
election or nomination for election was previously so approved; or

                  (v) one or more Acquiring Persons has succeeded, as the result
of or in response to actual or threatened election contests, whether by
settlement or otherwise, in having elected to the Board of Directors of the
Employer, whether at one time or on a cumulative basis, a sufficient number of
its nominees to constitute (a) more than thirty percent (30%) of the members of
the Employer's Board of Directors, rounded down to the nearest whole number, if
the number of directors on the Employer's Board is eight or less, or (b) more
than forty percent (40%) of the members of the Employer's Board, rounded down to
the nearest whole number, if the number of directors on the Employer's Board is
nine or more.

ARTICLE 7:  PAYMENT OF DEATH BENEFIT

         In the event of the death of the Employee while the Policy and this
Agreement are in force, the net proceeds of the Policy shall be divided into two
parts and paid as follows:

                  PART ONE: To the Employer, a sum equal to the aggregate amount
                  of premium payments for which the Employer was responsible
                  pursuant to Section A of Article 2 (less any outstanding
                  withdrawal by the Employer of the cash surrender value of the
                  Policy pursuant to Article 4, Section A), unless Section B of
                  Article 6 applies.

                  PART TWO: To the designated beneficiaries of the Employee, the
                  remaining proceeds of the Policy.

ARTICLE 8:  TERMINATION OF AGREEMENT

         This Agreement shall terminate:

         A. Upon surrender of the Policy by the Employee, except when the Policy
is surrendered and a new policy is issued pursuant to an exchange under Article
4, Section C.

         B. Upon the termination of the employment of the Employee for any
reason other than retirement at or after attaining the age of 65.

         C. On the death of the Employee.

ARTICLE 9:  EXCHANGE OF POLICY

         In the event the Employer is required to exchange the Policy under
Article 4, Section C, the Employee shall execute any forms necessary or
appropriate to effect such exchange including, without limitation, the surrender
of the Policy, the transfer of proceeds to the new insurer and the execution of
a new Split Dollar Agreement and Collateral Assignment. Such


                                      -5-

<PAGE>

exchange shall qualify under Section 1035 of the Internal Revenue Code or
successor provisions of similar import.

ARTICLE 10:  OBLIGATIONS OF INSURER

         Any payments made or action taken by the Insurer in accordance with the
provisions of the Policy and the Collateral Assignment shall fully discharge it
from all claims, suits, and demands of all persons whatsoever.

ARTICLE 11:  MISCELLANEOUS

         A. This Agreement shall be binding upon the parties hereto, their
heirs, legal representative, successors and assigns.

         B. This Agreement and the Collateral Assignment embody all agreements
between or among the parties with respect to the Policy, and no change,
alteration, or modification may be made except in writing signed by all parties
hereto. Without limiting the generality of the foregoing, this Agreement and the
Collateral Assignment supersede the Split Dollar Agreement dated as of November
18, 1999 and the related Assignment of Life Insurance Policy as Collateral dated
as of November 18, 1999 between the Employer and the Employee, which shall no
longer have any force or effect.

         C. This Agreement shall be governed by, and construed in accordance
with the provisions of, the laws of the Commonwealth of Massachusetts without
regard to its principles of conflicts of laws.

         D. Any dispute, controversy or claim with respect to any party's
performance under this Agreement shall be settled by arbitration in accordance
with the laws of The Commonwealth of Massachusetts by a single arbitrator who
shall be selected by the American Arbitration Association in Boston,
Massachusetts. Such arbitration shall be conducted in the City of Boston in
accordance with the Commercial Arbitration Rules of the American Arbitration
Association. Punitive damages shall not awarded. Judgment upon the award
rendered by the arbitrator may be entered in any court having jurisdiction
thereof.

         IN WITNESS WHEREOF, the parties hereto have set their hand and seals
effective as of the day and year first above written.


                                            /s/ Dennis Adomaitis, Employee
                                            --------------------
                                            Date: September 18, 2002


                                      -6-

<PAGE>

                                            THE J. JILL GROUP, INC., Employer

                                            By: /s/ Olga L. Conley
                                                ------------------
                                            Its CHIEF FINANCIAL OFFICER AND
                                                PRESIDENT - CORPORATE SERVICES

                                            Date: September 18, 2002


                                      -7-

<PAGE>

            SCHEDULE 1 TO AMENDED AND RESTATED SPLIT DOLLAR AGREEMENT

                ASSIGNMENT OF LIFE INSURANCE POLICY AS COLLATERAL


         A. FOR VALUE RECEIVED, the undersigned Dennis Adomaitis (hereinafter
the "Owner") hereby assigns, transfers and sets over to The J. Jill Group, Inc.,
a Delaware corporation, its successors and assigns (hereinafter the "Assignee"),
the following specific rights (and only those specific rights) in and to policy
number V062414100 issued by Allmerica Financial Life Insurance and Annuity
Company (hereinafter the "Insurer") and any supplementary contract or contracts
issued in connection therewith (said policy and any such contracts hereinafter
the "Policy"), insuring the life of the Owner, subject to all terms and
conditions of the Policy and to all superior liens, if any, which the Insurer
may have against the Policy. The Owner, by this Assignment, and the Assignee, by
acceptance of the Assignment of the Policy to it hereunder, agree to the terms
and conditions contained in the Policy.

         B. This Assignment is made, and the Policy is to be held as collateral
security for, all rights of and obligations owed to the Assignee, now existing
or hereafter arising under and pursuant to a certain Split Dollar Agreement, by
and between the Owner and the Assignee of even date herewith pertaining to the
Policy (hereinafter the "Agreement"). The Owner reserves all rights and powers
in and to the Policy, except those specific, limited rights in the Policy
granted to the Assignee hereby, as security for all rights of and obligations
owed to the Assignee under the Agreement.

         C. It is expressly agreed that the Assignee's interest in the Policy
under and by virtue of this Assignment shall be limited to the following
specific rights, and no others: (1) the right to withdraw all or part of the
cash surrender value as defined in the Policy in accordance with Article 4,
Section A of the Agreement, subject to the limitations and repayment obligations
set forth therein; (2) in the case of the death of the Owner, the right to be
paid the amounts due it under PART ONE of Article 7 of the Agreement by
recovering said amounts directly from the Insurer out of the net death proceeds
of the Policy; (3) in the event that Agreement is terminated pursuant to Article
8, Sections A or B, the right to be paid the amount due it, if any, under
Article 6 of the Agreement. Neither party shall have the right to borrow against
the Policy, except that the Owner may borrow against the Policy after attaining
the age of 65, so long as such borrowing shall not include funds from the
Assignee's interest in the Policy.

         D. Notwithstanding this Assignment, the Owner shall specifically retain
all incidents of ownership in and to the Policy, including, but not limited, to:
(1) in accordance with the terms of the Agreement, the right to cancel or
surrender the Policy and to receive, subject to Paragraph C. 2. hereof, the
surrender value thereof at any time provided by the terms of the Policy and at
such other times as the Insurer may allow; (2) the right to designate and change
the Policy beneficiary, with respect to the amount to be paid pursuant to PART
TWO of Article 7 of the Agreement; (3) the right to elect any optional methods
of settlement with regard to the death benefit under PART TWO of Article 7; (4)
the right to borrow against the Policy after the Owner attains age 65, so long
as such borrowing shall not include funds from the Assignee's interest in the
Policy; (5) the right to designate any reallocation of unit values as permitted
by the Policy subject to a right of the Assignee to disapprove a particular
allocation which it deems


                                      -8-

<PAGE>

inappropriate; and (6) all other rights contained in the Policy to the extent
the exercise of such rights does not adversely affect the interest of the
Assignee; provided, however, that all of the foregoing rights retained by the
Owner in the Policy shall be subject to the terms and conditions of the
Agreement.

         E. The Assignee agrees with the Owner as follows: (1) any funds
received by Assignee from the Insurer which are attributable to the portion of
the death benefit allocated to PART TWO of Article 7 of the Agreement shall be
paid by the Assignee to the beneficiaries designated by the Owner; and (2) if
the Policy is in the possession of the Assignee, the Assignee will, upon the
Owner's request, forward the Policy to the Insurer, without unreasonable delay,
for endorsement of any designation or change of beneficiary, any election of
optional mode of settlement, or the exercise of any other right reserved by the
Owner hereunder.

         F. Notwithstanding anything in this Assignment to the contrary, the
Insurer shall be under no obligation to monitor the obligation of the Assignee
hereunder to pay to the designated beneficiaries of the Owner any amounts
received from the Insurer under PART TWO of Article 7 of the Agreement after
payment of PART ONE to the Assignee under the Agreement; and the Insurer shall
have no obligation or liability to any person or entity if the Assignee fails to
pay such amounts as required hereunder.

         G. The Insurer is hereby authorized to recognize, and is protected in
recognizing, the Assignee's claims to amounts due it hereunder without
investigating the validity of its claim thereto, the reason for any action taken
by the Assignee, the validity or accuracy of the amount of any of the
liabilities of the Owner to the Assignee under the Agreement, the existence of
any default therein, the giving of any notice required therein, or the
application to be made by the Assignee of any amounts to be paid to the
Assignee. A receipt executed solely by the Assignee for any amounts received by
it from the Insurer shall be a full discharge and release of the Insurer from
the obligations released thereby.

         H. In furtherance hereof, the Owner appoints the Assignee his
attorney-in-fact for the following purposes:

         1.    to receive the portion of the death benefit payable to the
               Assignee upon the death of the Owner under PART ONE of Article 7
               of the Agreement; and

         2.    to receive, upon termination of the Agreement pursuant to Article
               8, Sections A or B thereof, the amount, if any, designated in
               Article 6, Sections A or B of the Agreement, as the case may be.

         This appointment is coupled with an interest in the Assignee and shall
be irrevocable so long as the Agreement remains in force.

         I. The Insurer shall not comply with a request made by the Owner for
cancellation or surrender of the Policy without the consent of the Assignee.
Upon receipt of an assented-to request for cancellation or surrender, the
Insurer shall terminate the Policy and this Assignment shall be of no further
force or effect; provided, however, that the Insurer shall set aside and


                                      -9-

<PAGE>

deduct from any amounts to be paid to the Owner under the terms of the Policy in
consequence of its cancellation or surrender, the amount due to the Assignee
under the provisions of Article 6 of the Agreement, which amount shall be paid
to the Assignee by the Insurer.

         J. In the event of any conflict between the provisions of this
Assignment and the provisions of the Agreement with respect to the Policy or the
Assignee's rights therein, the provisions of the Agreement shall prevail.

         K. The Owner declares that no proceedings in bankruptcy are pending
against the Owner and that the Owner's property is not subject to any assignment
for the benefit of creditors of the Owner.

         L. This Assignment supersedes the Assignment of Life Insurance Policy
As Collateral dated as of November 18, 1999 by Owner in favor of Assignee with
respect to the Policy, which shall no longer have any force or effect.

         Signed and sealed as of the 18th day of September, 2002.


                                                    /s/ Dennis Adomaitis, Owner
                                                    --------------------
                                                    Date:  September 18, 2002

Accepted and Agreed:

The J. Jill Group, Inc.

By: /s/ Olga L. Conley
    ------------------
    Its: CHIEF FINANCIAL OFFICER AND PRESIDENT OF CORPORATE SERVICES

Date:  September 18, 2002


                                      -10-

<PAGE>

                      ACKNOWLEDGEMENT OF SIGNATURE OF OWNER

COMMONWEALTH OF MASSACHUSETTS)
COUNTY OF NORFOLK       ) ss:


On the 18th day of September, 2002, before me personally came Dennis Adomaitis,
to me known to be the individual described in and who executed the assignment
above and acknowledged to me that he/she executed the same.

                                         Notary Public: /s/ Susan Galvin
                                                        -----------------------
                                         My commission expires: April 25, 2008





                    ACKNOWLEDGEMENT OF SIGNATURE OF ASSIGNEE

COMMONWEALTH OF MASSACHUSETTS)
COUNTY OF NORFOLK       ) ss:


On the 18th day of September, 2002, before me personally came Olga Conley, who
being by me duly sworn, did depose and say that he/she is the Chief Financial
Officer and President of Corporate Services of The J. Jill Group, Inc., the
corporation described in and which executed the acceptance and agreement of
assignment above; and that he/she signed his/her name thereto by the authority
granted to his/her office.

                                         Notary Public: /s/ Susan Galvin
                                                        -----------------------
                                         My commission expires: April 25, 2008


                                      -11-