<PAGE>
                                                                   Exhibit 10.12

                                FOURTH AMENDMENT
                                       TO
           SEVENTH AMENDED AND RESTATED PLEDGE AND SECURITY AGREEMENT



         THIS FOURTH AMENDMENT TO SEVENTH AMENDED AND RESTATED PLEDGE AND
SECURITY AGREEMENT (the "Amendment") made and entered into as of the 26th day of
November, 2003, by and between DIRECT GENERAL CORPORATION, a Tennessee
corporation (the "Pledgor"), and FIRST TENNESSEE BANK NATIONAL ASSOCIATION,
whose address is 165 Madison Avenue, Memphis, Tennessee 38103, Attention:
Metropolitan Division (in its agency capacity being herein referred to as
"Agent," and in its individual capacity as "FTBNA"), as agent for itself, and
for HIBERNIA NATIONAL BANK, Baton Rouge, Louisiana ("Hibernia"), and for U. S.
BANK NATIONAL ASSOCIATION, St. Louis, Missouri ("U. S. Bank"), and for REGIONS
BANK, Birmingham, Alabama ("Regions"), and for CAROLINA FIRST BANK, Greenville,
South Carolina ("Carolina First"), and for BANK ONE, NA (Main Office -- Chicago,
Illinois), Baton Rouge, Louisiana ("Bank One"), and for NATIONAL CITY BANK OF
KENTUCKY, Louisville, Kentucky ("National City Bank"), and for FIFTH THIRD BANK,
Franklin, Tennessee ("Fifth Third") (Agent, Hibernia, U. S. Bank, Regions,
Carolina First and Bank One collectively, the "Original Banks") (the Original
Banks, Fifth Third and National City Bank collectively, the "Banks" and
individually, a "Bank").

                                Recitals of Fact

         Pursuant to that certain Seventh Amended and Restated Pledge and
Security Agreement dated as of October 31, 2002 (the "Original Pledge
Agreement"), Pledgor pledged a second lien security interest in all of the stock
in the Agency Subsidiaries and Affiliated Insurers (as those terms are defined
in the Eighth Amended and Restated Loan Agreement dated as of October 31, 2002,
as amended, referred to hereafter as the "Loan Agreement") to the Agent for the
benefit of the Original Banks as security for all of Pledgor's obligations under
the Eighth Amended and Restated Guaranty Agreement dated October 31, 2002 (the
"Eighth Guaranty").

         Pursuant to that certain First Amendment to Seventh Amended and
Restated Pledge and Security Agreement dated as of March 31, 2003 (the "First
Amendment"), Pledgor pledged a second lien security interest in all of the stock
in the Agency Subsidiaries and Affiliated Insurers (as those terms are defined
in the Loan Agreement) to the Agent for the benefit of the Original Banks as
security for all of Pledgor's obligations under the Ninth Amended and Restated
Guaranty Agreement dated March 31, 2003 (the "Ninth Guaranty").

         Pursuant to that certain Second Amendment to Seventh Amended and
Restated Pledge and Security Agreement dated as of May 28, 2003 (the "Second
Amendment"), Pledgor pledged a second lien security interest in all of the stock
in the Agency Subsidiaries and Affiliated Insurers (as those terms are defined
in the Loan Agreement) to the Agent for the benefit of the Original Banks and
National City Bank as security for all of Pledgor's obligations under the Tenth
Amended and Restated Guaranty Agreement dated May 28, 2003 (the "Tenth
Guaranty").

         Pursuant to that certain Third Amendment to Seventh Amended and
Restated Pledge and Security Agreement dated as of June 30, 2003 (the "Third
Amendment" and the Original Pledge Agreement, the First Amendment, the Second
Amendment and the Third Amendment collectively referred to herein as the "Pledge
Agreement"), Pledgor pledged a second lien security interest in all of the stock
in the Agency Subsidiaries and Affiliated Insurers (as those terms are defined
in the Loan Agreement) to the Agent for the benefit of the Banks as security for
all of Pledgor's obligations under the Eleventh Amended and Restated Guaranty
Agreement dated June 30, 2003 (the "Eleventh Guaranty").

         Pledgor is entering into a Twelfth Amended and Restated Guaranty
Agreement of even date herewith (the "Twelfth Guaranty"), which Twelfth Guaranty
replaces the Eleventh Guaranty.

         The Pledgor and the Banks now desire to modify certain terms of the
Pledge Agreement as hereinafter set forth.
<PAGE>

         NOW, THEREFORE, in consideration of the premises as set forth in the
Recitals of Fact, the mutual covenants and agreements hereinafter set out, and
other good and valuable consideration, the receipt and sufficiency of which are
hereby acknowledged, it is agreed by the parties as follows:

                                   Agreements

         1.       All capitalized terms used and not defined herein shall have
the meaning ascribed to them in the Loan Agreement.

         2.       To induce the Banks to enter into this Amendment, the Pledgor
does hereby absolutely and unconditionally, certify, represent and warrant to
the Banks, and covenants and agrees with the Banks, that:

                  (a)      All representations and warranties made by the
         Pledgor in the Loan Agreement, as amended, in the Pledge Agreement, as
         amended, in the Security Agreement, as amended, and in all other loan
         documents (all of which are herein sometimes called the "Loan
         Documents"), are true, correct and complete in all material respects as
         of the date of this Amendment.

                  (b)      As of the date hereof and with the execution of this
         Amendment, there are no existing events, circumstances or conditions
         which constitute, or would, with the giving of notice, lapse of time,
         or both, constitute Events of Default.

                  (c)      There are no existing offsets, defenses or
         counterclaims to the obligations of the Pledgor, as set forth in the
         Pledge Agreement, as amended, the Security Agreement, as amended, the
         Loan Agreement, as amended, or in any other Loan Document executed by
         the Borrower, in connection with the Loan.

                  (d)      The Pledgor does not have any existing claim for
         damages against the Banks arising out of or related to the Loan; and,
         if and to the extent (if any) that the Borrower has or may have any
         such existing claim (whether known or unknown), the Borrower does
         hereby forever release and discharge, in all respects, the Banks with
         respect to such claim.

                  (e)      The Loan Documents, as amended by this Amendment, are
         valid, genuine, enforceable in accordance with their respective terms,
         and in full force and effect.

         3.       Section 2(a) of the Pledge Agreement, is hereby deleted in its
entirety and the following is inserted in lieu thereof:

                  (a)      the prompt payment by the Pledgor, as and when due
         and payable, of all amounts from time to time owing under or pursuant
         to that certain Twelfth Amended and Restated Guaranty Agreement (the
         "Guaranty Agreement") dated as of November 26, 2003, pursuant to which
         Pledgor has guaranteed the indebtednesses of Direct General Financial
         Services, Inc., a Tennessee corporation ("Borrower"), to FTBNA,
         Hibernia, U.S. Bank, Regions, Carolina First, Fifth Third, National
         City Bank and Bank One, which indebtednesses are evidenced by (i)
         Borrower's promissory note dated as of November 26, 2003, to FTBNA in
         the maximum principal amount of Forty Million Dollars ($40,000,000.00);
         (ii) Borrower's promissory note dated as of November 26, 2003, to
         Hibernia in the maximum principal amount of Twenty Million Dollars
         ($20,000,000.00), (iii) Borrower's promissory note dated as of November
         26, 2003, to U.S. Bank in the maximum principal amount of Thirty
         Million Dollars ($30,000,000.00); (iv) Borrower's promissory note dated
         as of November 27, 2003, to Regions in the maximum principal amount of
         Twenty-Five Million Dollars ($25,000,000.00); (v) Borrower's promissory
         note dated as of November 26, 2003, to Carolina First in the maximum
         principal amount of Fifteen Million Dollars ($15,000,000.00); (vi)
         Borrower's promissory note dated as of November 26, 2003, to National
         City Bank in the maximum principal amount of Fifteen Million Dollars
         ($15,000,000.00); (vii) Borrower's promissory note dated as of November
         26, 2003, to Fifth Third


                                       2
<PAGE>

         in the maximum principal amount of Ten Million Dollars
         ($10,000,000.00); and (viii) Borrower's promissory note dated as of
         November 26, 2003, to Bank One in the maximum principal amount of
         Thirty-Five Million Dollars ($35,000,000.00) [the eight (8) promissory
         notes herein described being collectively referred to as the "Notes,"
         and FTBNA, Hibernia, U.S. Bank, Regions, Carolina First, National City
         Bank, Fifth Third and Bank One being collectively referred to as the
         "Banks"]; and

         4.       Section 2(b) of the Pledge Agreement, is hereby deleted in its
entirety and the following is inserted in lieu thereof:

                  (b)      the due performance and observance by the Pledgor of
         all of the Pledgor's covenants, agreements, duties, representations and
         obligations from time to time existing pursuant to this Agreement and
         that certain Eighth Amended and Restated Loan Agreement dated October
         21, 2002, as amended by that certain First Amendment to Eighth Amended
         and Restated Loan Agreement dated as of March 31, 2003, as amended by
         that certain Second Amendment to Eighth Amended and Restated Loan
         Agreement dated as of May 28, 2003, as amended by that certain Third
         Amendment to Eighth Amended and Restated Loan Agreement dated as of
         June 30, 2003, as amended by that certain Fourth Amendment to Eighth
         Amended and Restated Loan Agreement dated on or about July 17, 2003,
         and as amended by that certain Fifth Amendment to Eighth Amended and
         Restated Loan Agreement dated as of November 26, 2003 (as amended, the
         "Loan Agreement"), among Pledgor, Borrower, Agent, the Banks and
         others; and in any other instrument which now or hereafter secures the
         Guaranty Agreement.

         5.       All terms and provisions of the Pledge Agreement, as
heretofore amended, which are inconsistent with the provisions of this Amendment
are hereby modified and amended to conform hereto; and, as so modified and
amended, the Pledge Agreement is hereby ratified, approved and confirmed. Except
as otherwise may be expressly provided herein, this Amendment shall become
effective as of the date set forth in the initial paragraph hereof.

         6.       All references in all Loan Documents (including, but not
limited to, the Pledge Agreement, the Security Agreement, and the Loan
Agreement) to the "Pledge Agreement" shall, except as the context may otherwise
require, be deemed to constitute references to the Pledge Agreement as amended
hereby.

                        [SEPARATE SIGNATURE PAGES FOLLOW]


<PAGE>

                                 SIGNATURE PAGE
                                       TO
 FOURTH AMENDMENT TO SEVENTH AMENDED AND RESTATED PLEDGE AND SECURITY AGREEMENT

================================================================================


         IN WITNESS WHEREOF, the Pledgor, the Banks and the Agent have caused
this Agreement to be executed by their duly authorized officers, all as of the
day and year first above written.

                                   PLEDGOR:

                                   DIRECT GENERAL CORPORATION,
                                   a Tennessee corporation

                                   By:  /s/ Barry D. Elkins
                                      -----------------------------------------
                                   Title:  SVP & CFO
                                         --------------------------------------



                                   BANKS:

                                   FIRST TENNESSEE BANK NATIONAL ASSOCIATION

                                   By:  /s/ Sam Jenkins
                                      -----------------------------------------
                                   Title:  SVP
                                         --------------------------------------

                                   HIBERNIA NATIONAL BANK

                                   By:  /s/ Janet Olson Rack
                                      -----------------------------------------
                                   Title:  Senior Vice President
                                         --------------------------------------

                                   U. S. BANK NATIONAL ASSOCIATION

                                   By:  /s/ Russell S. Rogers
                                      -----------------------------------------
                                   Title:  Vice President
                                         --------------------------------------

                                   CAROLINA FIRST BANK

                                   By:  /s/ Charles Chamberlin
                                      -----------------------------------------
                                   Title:  Executive Vice President
                                         --------------------------------------

                           [SIGNATURE PAGE CONTINUED]


                                      S-1
<PAGE>

                                    BANK ONE, NA
                                    (MAIN OFFICE - CHICAGO, ILLINOIS)

                                    By:  /s/ Robert D. Bond
                                       ----------------------------------------
                                    Title:  First Vice President
                                          -------------------------------------

                                    REGIONS BANK

                                    By:  /s/ Sam Prudhomme
                                       ----------------------------------------
                                    Title:  Assistant Vice President
                                          -------------------------------------

                                    NATIONAL CITY BANK OF KENTUCKY

                                    By:  /s/ Kevin C. Anderson
                                       ----------------------------------------
                                    Title:  SVP
                                          -------------------------------------

                                    FIFTH THIRD BANK

                                    By:  /s/ David Hicks
                                       ----------------------------------------
                                    Title:  VP
                                          -------------------------------------



                                    AGENT:

                                    FIRST TENNESSEE BANK NATIONAL ASSOCIATION

                                    By:  /s/ Sam Jenkins
                                       ----------------------------------------
                                    Title:  SVP
                                          -------------------------------------




                                      S-2