<PAGE>   1
                                                                    EXHIBIT 10.7

                                                                [EXECUTION COPY]


                                 AMENDMENT NO. 4
                                       to
                              Amended and Restated
                           Loan and Security Agreement
                          dated as of December 20, 1996

         AMENDMENT NO. 4 entered into as of March 13, 1998 among RIDGEVIEW,
INC., a North Carolina corporation (for itself and as successor by merger to
InterKnit, Inc., an Alabama corporation), SENECA KNITTING MILLS CORPORATION, a
New York corporation (collectively, the "Borrowers"), and NATIONSBANK, N.A., a
national banking association (the "Lender").

                              Preliminary Statement

         The Borrowers and the Lender are parties to that certain Amended and
Restated Loan and Security Agreement dated as of December 20, 1996, as amended
by Amendment No.1 dated as of January 31, 1997, Amendment No.2 dated as of March
13, 1997 and Amendment No. 3 dated as of July 31, 1997 (the "Loan Agreement,"
terms defined therein, unless otherwise defined herein, being used herein as
therein defined).

         The Borrower has requested an extension of the maturity of the
Revolving Credit Facility, a reduction in the interests rates applicable to
Loans, and certain other modifications to the Loan Agreement, and the Lender has
agreed to such request, upon and subject to all of the terms, conditions and
provisions hereinafter set forth.

         NOW, THEREFORE, in consideration of the Loan Agreement, the mutual
covenants set forth therein and herein and other good and valuable
consideration, the receipt and sufficiency of which are hereby acknowledged, the
parties hereto hereby agree as follows:

         Section 1. Amendment to Loan Agreement. Subject to the provisions of
Section 2, the Loan Agreement is hereby amended:

         (a) by amending the provisions of Section 1.1 Definitions as follows:

                  (i) by amending the definition "Applicable Margin" in its
         entirety to read as follows:

                           "Applicable Margin" means (a) from and after March 1,
                  1998, as to Prime Rate Loans -0-% and (b) as to Eurodollar
                  Rate Loans made on or after March 1, 1998, the rate per annum
                  set forth below opposite the Funded Debt to EBITDA Ratio
                  reflected in the financial statements of the Borrowers
                  delivered to the Lender pursuant to Section 9.1(b) for the
                  then most recently ended period of four


<PAGE>   2

                  consecutive fiscal quarters (provided such financial
                  statements shall have been delivered to the Lender at least
                  five Business Days prior to the determination of the
                  Applicable Margin):

<TABLE>
<CAPTION>
                  Funded Debt to
                  EBITDA Ratio                                Applicable Margin

                                              Prime Rate       Eurodollar Rate
                                                 Loans              Loans
                                                 -----              -----

<S>                                           <C>              <C>
                  < 2.50 to 1                      0%                1.75%
                  =>2.50 to 1 <2.75 to 1           0%                1.87%
                  =>2.75 to 1 <3.00 to 1           0%                2.00%
                  =>3.00 to 1 <3.25 to 1           0.12%             2.12%
                  =>3.25 to 1 <3.50 to 1           0.25%             2.25%
                  =>3.50 to 1                      0.37%             2.37%
</TABLE>

                  "Funded Debt to EBITDA Ratio" means, as of the last day of any
                  fiscal quarter of the Borrowers, the ratio of Funded Debt as
                  of such last day to EBITDA for the period of four consecutive
                  fiscal quarters ended on such last day; "Funded Debt" means
                  the aggregate outstanding principal amount of all Indebtedness
                  for Money Borrowed of the Borrowers computed on a consolidated
                  basis; and "EBITDA" for any specified accounting period means
                  Net Income of the Borrowers on a consolidated basis for such
                  period, plus the sum of the amounts deducted in respect of
                  interest expense, income taxes, depreciation and amortization
                  in computing such Net Income. In the event that financial
                  statements of the Borrowers have not been timely delivered in
                  accordance with Section 9.1(b) and, therefore, the Funded Debt
                  to EBITDA Ratio cannot be determined, the Applicable Margin
                  shall be equal to the sum of the then effective Applicable
                  Margin and the Default Rate.

                  (ii) by amending the definition "Borrowing Base" in its
         entirety to read as follows:

                  "Borrowing Base" means at any time an amount equal to the sum
         of:

                          (a) 80% (or such lesser percentage as the Lender may
                  in its sole and absolute discretion determine from time to
                  time) of the face value of Eligible Receivables due and owing
                  at such time, plus


                                       2
<PAGE>   3

                           (b) the lesser of

                                    (i) subject to the proviso below, 50% (or
                            such lesser percentage as the Lender may in its sole
                            and absolute discretion determine from time to time)
                            of the lesser of cost (computed on a
                            first-in-first-out basis) and fair market value of
                            Eligible Inventory at such time, and

                                    (ii) $12,500,000, minus

                           (c) the Letter of Credit Obligations and such other
                  reserves as the Lender may determine from time to time in the
                  exercise of its reasonable credit judgment;

                  provided, that the stated advance rate (subject to the
                  Lender's discretion) against Eligible Inventory may be 60% for
                  not more than 120 consecutive days during the period May 1
                  through September 30 of each calendar year and shall not be
                  greater than 40% during a period of at least 60 consecutive
                  days during the period October 1 of one calendar year through
                  February 28 (or 29) of the following calendar year. Each
                  period during which such stated advance rate may be 60% shall
                  commence on the date of delivery of a Borrowing Base
                  Certificate during the relevant period reflecting either an
                  advance rate against Eligible Inventory of 60% or a deficit in
                  Collateral Availability that would be cured by application of
                  a 60% advance rate; each period during which such stated
                  advance rate shall not exceed 40% shall commence on the date
                  of delivery of a Borrowing Base Certificate during the
                  relevant period reflecting a 40% advance rate against Eligible
                  Inventory, but in any event not later than January 1 of each
                  calendar year.

                  (iii) by amending the definition "CAPEX Lock-in Date" in its
         entirety to read as follows:

                  "CAPEX Lock-in Date" means June 30, 1998.

                  (iv) by amending the definition "Revolving Credit Facility" by
         deleting the figure "$18,000,000" appearing therein and substituting
         therefor the figure "$28,000,000";

                  (v) by amending the definition "Termination Date" by deleting
         the date "January 10, 1999" appearing therein and substituting therefor
         the date "June 30, 2000";

         (b) by further amending Section 1.1 Definitions by adding thereto in
appropriate alphabetical order the following additional definitions:



                                       3
<PAGE>   4

                  "Collateral Availability" means the excess, if any, of the
         Borrowing Base as in effect on the date of determination, over the
         aggregate outstanding principal amount of Revolving Credit Loans on the
         same date.

                  "Fiscal Year" means the calendar year and when followed or
         preceded by a designated year, means such calendar year.

         (c) by amending Section 2B.2 Repayment of Term Loan in its entirety to
read as follows;

                  Section 2B.2 Repayment of Term Loan. The Term Loan is due and
         payable, and shall be repaid in full by the Borrowers in 43
         installments as follows: the first 42 installments, payable on January
         1, 1997 and on the first day of each calendar month thereafter, shall
         be in the amount of $53,667 each and the final installment payable on
         June 30, 2000 shall be in the amount of the then unpaid balance of the
         Term Loan.

         (d) by amending Section 2C.4 Repayment of CAPEX Loan in its entirety to
read as follows:

                  Section 2C.4 Repayment of CAPEX Loan. The CAPEX Loan is due
         and payable, and shall be repaid in full by the Borrowers in 24
         substantially equal consecutive monthly installments payable on the
         first day of each month beginning July 1, 1998 through and including
         June 1, 2000, each in an amount equal to 1/84th of the aggregate
         principal balance of the CAPEX Loans outstanding on the CAPEX Lock-in
         Date, and a 25th and final installment in the amount of the entire
         remaining principal balance, payable on June 30, 2000.

         (e) by amending Section 10.5 Capital Expenditures in its entirety to
read as follows:

                  Section 10.5 Capital Expenditures. Make or incur any Capital
         Expenditures, except that Ridgeview and its Subsidiaries may make or
         incur Capital Expenditures in Fiscal Year 1997 not exceeding
         $4,100,000, in Fiscal Year 1998 not exceeding $4,000,000 and in any
         Fiscal Year thereafter, not exceeding $1,000,000.

         (f) by further amending the Loan Agreement by substituting one of the
Schedules attached hereto as a part of Annex 1 for the Schedule bearing the same
caption attached to the Loan Agreement on the Amendment Effective Date; and

         (g) by amending Exhibit E (Form of Compliance Certificate) to the Loan
Agreement in its entirety to read in the form of Exhibit E attached hereto as
Annex 2.

         Section 2. Effectiveness. The provisions of Section 1 shall become
effective as of the date hereof on the date (the "Amendment Effective Date") on
which the Lender shall have received each of the following, in form and
substance satisfactory to the Lender:

                                       4
<PAGE>   5

         (a) payment by the Borrowers of a $42,500 origination fee with respect
to the increase in the Revolving Credit Facility effected by this Amendment
(which amount the Borrowers hereby irrevocably authorize the Lender to charge to
any account of the Borrowers (or an individual Borrower)) maintained with the
Lender;

         (b) counterparts of this Amendment duly executed by the Borrowers,
together with the Consent and Confirmation attached hereto duly executed by the
Guarantor;

         (c) a Revolving Credit Note in the original principal amount of
$28,000,000, duly executed and delivered by each Borrower;

         (d) a certificate of the chief operating officer or president of each
Borrower stating that, to the best of his knowledge and based on an examination
sufficient to enable him to make an informed statement:

                  (i) all of the representations and warranties made or deemed
         to be made under the Loan Agreement are true and correct as of the
         Amendment Effective Date, and

                  (ii) after giving effect to this Amendment, no Default or
         Event of Default exists;

         (e) a certificate of the chief financial officer of Ridgeview,
accompanied by supporting calculations of the Funded Debt to EBITDA Ratio (as
defined in the Loan Agreement as amended by this Amendment) in reasonable
detail, setting forth the Applicable Margins effective as of the Amendment
Effective Date, based on September 30, 1997 financial statements and EBITDA for
the four-quarter period ended on that date;

         (f) a signed opinion of Isenhower, Wood & Cilley, P.A., counsel for the
Borrowers, and such local counsel as the Lender shall deem necessary and
desirable, opining as to such matters in connection with this Amendment as the
Lender or its counsel may reasonably request; and

         (g) such other documents and instruments as the Lender may reasonably
request.

         Section 3. Effect of Amendment. Upon and after the effectiveness of
this Amendment as provided in Section 2 hereof, all references to the Loan
Agreement in the Loan Agreement or in any other Loan Document shall mean the
Loan Agreement as amended by this Amendment. Except as expressly provided in
this Amendment, the execution and delivery of this Amendment does not, and will
not, amend, modify or supplement any provision of or constitute a consent to or
a waiver of any noncompliance with the provisions of the Loan Agreement and,
except as specifically provided in this Amendment, the Loan Agreement shall
remain in full force and effect.


                                       5
<PAGE>   6

         Section 4. Representations and Warranties. Each Borrower hereby makes
the following representations and warranties to the Lender, which
representations and warranties shall survive the delivery of this Amendment and
the making of Loans under the Loan Agreement as amended hereby:

         (a) Organization; Power; Qualification. Each Borrower is a corporation,
duly organized, validly existing and in good standing under the laws of the
jurisdiction of its incorporation, has the corporate power and authority to own
its properties and to carry on its business as now being and hereafter proposed
to be conducted and is duly qualified and authorized to do business in each
jurisdiction in which failure to be so qualified and authorized would have a
Materially Adverse Effect.

         (b) Authorization of Agreements. Each Borrower has the right and power
and has taken all necessary action to authorize it to execute, deliver and
perform this Amendment and the other Loan Documents contemplated hereby in
accordance with its terms. This Amendment has been, and the other Loan Documents
contemplated hereby when executed and delivered by the Borrowers, will have
been, duly executed and delivered by the duly authorized officers of each
Borrower and is, or at such time will be, a legal, valid and binding obligation
of such Borrower, enforceable in accordance with its terms.

         (c) Compliance of Agreements with Laws. The execution and delivery of
this Amendment and the other Loan Document contemplated hereby, and the
performance of the Loan Agreement as amended by this Amendment and of the other
Loan Documents in accordance with their respective terms, do not and will not,
by the passage of time, the giving of notice or otherwise,

                  (i) require any Governmental Approval or violate any
         applicable law relating to such Borrower or any of its Affiliates,

                  (ii) conflict with, result in a breach of or constitute a
         default under (1) the articles of incorporation or by-laws or any
         shareholders' agreement of such Borrower, (2) any indenture, agreement
         or other instrument to which such Borrower is a party or by which any
         of its property may be bound or (3) any Governmental Approval relating
         to such Borrower, or

                  (iii) result in or require the creation or imposition of any
         Lien upon or with respect to any property now owned or hereafter
         acquired by such Borrower other than the Security Interest.

         Section 5.  General Provisions.

         (a) Expenses. (i) The Borrowers agree to pay or reimburse on demand all
costs and expenses incurred by the Lender, including, without limitation, the
reasonable fees and



                                       6
<PAGE>   7

disbursements of counsel, in connection with (a) the negotiation, preparation,
execution, delivery, administration, enforcement and termination of this
Amendment, the Loan Agreement and each of the other Loan Documents, whenever the
same shall be executed and delivered, including, without limitation, the
out-of-pocket costs and expenses incurred in connection with the administration
and interpretation of this Amendment, the Loan Agreement and the other Loan
Documents, (b) sums paid or obligations incurred in connection with the payment
of any amount or taking any action required of a Borrower under this Amendment,
the Loan Agreement and the other Loan Documents that such Borrower fails to pay
or take, and (c) any other obligations or expenses of the Borrowers under
Section 12.2 of the Loan Agreement.

         (ii) The foregoing shall not be construed to limit any other provisions
of the Loan Documents regarding costs and expenses to be paid by the Borrowers.
The Borrowers hereby irrevocably authorize the Lender to charge to any account
of the Borrowers (or any individual Borrower) maintained with the Lender in the
amount of any costs and expenses owed by the Borrowers , as set forth in this
Section or otherwise, when such costs and expenses are due to the Lender and to
deem such amounts (but not in duplication) a request for a borrowing under the
Revolving Credit Facility pursuant to the provisions of Section 2A.2(a)(iii) of
the Loan Agreement.

         (b) Governing Law. This Amendment shall be construed in accordance with
and governed by the law of the State of Georgia.

         (c) Counterpart Execution. This Amendment may be executed in any number
of counterparts and by different parties hereto in separate counterparts, each
of which when so executed shall be deemed to be an original and shall be binding
upon all parties, their successors and assigns, and all of which taken together
shall constitute one and the same agreement.


                                       7
<PAGE>   8


         IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
executed and delivered by their respective officers thereunto duly authorized as
of the date first above written.

                                              RIDGEVIEW, INC.

[CORPORATE SEAL]

Attest:                                       By: /s/ Hugh R. Gaither
                                                  ----------------------------
                                                  Hugh R. Gaither
                                                  President
By: /s/ Doug Yoder
    ------------------------------
    Name:  Doug Yoder
    Title: Assistant Secretary

                                              SENECA KNITTING MILLS
CORPORATION

[CORPORATE SEAL]

Attest:                                       By: /s/ Hugh R. Gaither
                                                  ----------------------------
                                                  Hugh R. Gaither
                                                  President
By: /s/ Walter L. Bost Jr.
    ------------------------------
    Name:  Walter L. Bost Jr.
    Title: Secretary




                                              NATIONSBANK, N.A.


                                              By: /s/ Scott K. Goldstein
                                                  ----------------------------
                                                  Scott K. Goldstein
                                                  Vice President


                                       8
<PAGE>   9

                            CONSENT AND CONFIRMATION

         The undersigned, GPM Corporation, as the maker of the Subsidiary
Guaranty, hereby acknowledges receipt of the foregoing Amendment No. 4 and
confirms, for the benefit of the Borrowers and the Lender, that the Subsidiary
Guaranty remains in full force and effect, in accordance with its terms, as to
Secured Obligations of the Borrower under the Loan Agreement as amended by the
said Amendment No. 4 and is hereby in all respects ratified and confirmed.

Dated:  As of March 16, 1998

                                              GPM CORPORATION

                                              By: /s/ Hugh R. Gaither
                                                  ----------------------------
                                                  Hugh R. Gaither
                                                  President


                                       9
<PAGE>   10

                                                                         ANNEX 1


                     Revised Schedules to the Loan Agreement


                                      NONE




                                       10
<PAGE>   11

                                                                         ANNEX 2

                                                                       EXHIBIT E

                         FORM OF COMPLIANCE CERTIFICATE


                  The undersigned, ______________________,
__________________________ of each of RIDGEVIEW, INC., a North Carolina
corporation, and SENECA KNITTING MILLS CORPORATION, a New York corporation
(collectively, the "Borrowers"), hereby certifies to NATIONSBANK, N.A., f/k/a
NationsBank, N.A. (South) (the "Lender") under the Amended and Restated Loan and
Security Agreement dated as of December 20, 1996 (as amended and in effect on
the date hereof, the "Loan Agreement") between the Borrowers and the Lender, in
accordance with the provisions of Section 9.3 of the Loan Agreement, that:

         1. As of __________________ ___, 199__/20__, [date of last day of
fiscal month/fiscal year], the Borrowers were/were not in compliance with the
covenants set forth in Sections 10.1, 10.2 and 10.5 of the Loan Agreement, as
detailed on the worksheet attached.

         2. As of __________________ ___, 199__/20__, the Funded Debt to EBITDA
Ratio was ______ to 1; as a result, the Applicable Margin for Prime Rate Loans
is __% and for Eurodollar Rate Loans is __%.

         3. Based on an examination sufficient to enable me to make an informed
statement, [no Default or Event of Default exists as of the date hereof] [The
following Defaults and Events of Default exist [describe nature of Default or
Event of Default, when it occurred, whether it is continuing and the steps the
applicable Borrower is taking with respect thereto]].

                  IN WITNESS WHEREOF, the undersigned in his aforesaid capacity
has executed and delivered this Certificate as of __________ ___, 199__.



                                     -------------------------------------
                                     Name:
                                     Title:



                                       11


<PAGE>   12
                              AMENDED AND RESTATED
                             REVOLVING CREDIT NOTE



$28,000,000.00                                                  Atlanta, Georgia
                                                                  March 13, 1998




          FOR THE VALUE RECEIVED, the undersigned, RIDGEVIEW, INC., a North
Carolina corporation ("Ridgeview"), and SENECA KNITTING MILLS CORPORATION, a
New York corporation (collectively, the "Borrowers"), hereby jointly and
severally unconditionally promise to pay to the order of NATIONSBANK, N.A.
(f/k/a/ NationsBank, N.A. (South), the "Lender"), at the offices of the Lender
located at 600 Peachtree Street, N.E., Atlanta, Georgia, 30308, or at such
other place within the United States as shall be designated from time to time
by the Lender, on the Termination Date, the principal amount of TWENTY-EIGHT
MILLION AND NO/100 DOLLARS ($28,000,000.00), or such lesser principal amount as
may then constitute the aggregate unpaid balance of all Revolving Credit Loans
made by the Lender to the Borrowers pursuant to the Loan Agreement, in lawful
money of the United States of America in federal or other immediately available
funds.

          The Borrowers also jointly and severally unconditionally promise to
pay interest on the unpaid principal amount of this Note outstanding from time
to time for each day from the date of disbursement until such principal amount
is paid in full at the rates per annum and on the dates specified in the Loan
Agreement applicable from time to time in accordance with the provisions
thereof.  Nothing contained in this Note or in the Loan Agreement shall be
deemed to establish or require the payment of a rate of interest in excess of
the maximum rate permitted by any applicable law.  In the event that any rate
of interest required to be paid hereunder exceeds the maximum rate permitted by
applicable law, the provisions of the Loan Agreement relating to the payment of
interest under such circumstances shall control.

          This Note is the Revolving Credit Note referred to in that certain
Amended and Restated Loan and Security Agreement dated as of December 20, 1996
(as amended, modified, supplemented or restated from time to time, the "Loan
Agreement"; terms defined therein being used in this Note as therein defined)
between the Borrowers and the Lender, is subject to, and entitled to, all
provisions and benefits of the Loan Documents, is secured by the Collateral and
other property as provided in the Loan Documents, is subject to optional and
mandatory prepayment in whole or in part and is subject to acceleration prior
to maturity upon the occurrence of one or more Events of Default, all as
provided in the Loan Documents.

<PAGE>   13
          Presentment for payment, demand, protest and notice of demand, notice
of dishonor, notice of non-payment and all other notices are hereby waived by
the Borrowers, except to the extent expressly provided in the Loan Agreement.
No failure to exercise, and no delay in exercising, any right hereunder on the
part of the holder hereof shall operate as a waiver of such rights.

          This Amended and Restated Revolving Credit Note is delivered in
exchange and substitution for, but not in extinguishment of the indebtedness
outstanding under, that certain Revolving Credit Note dated December 20, 1996
in the original principal amount of $14,000,000 made by the Borrowers in favor
of the Lender, as increased to $18,000,000 pursuant to Amendment No. 3 dated as
of July 31, 1997.

          The Borrowers hereby jointly and severally agree to pay on demand all
costs and expenses incurred in collecting the Secured Obligations hereunder or
in enforcing or attempting to enforce any of the Lender's rights hereunder,
including, but not limited to, reasonable attorneys' fees and expenses if
collected by or through an attorney, whether or not suit is filed.

          THE PROVISIONS OF SECTION 12.5 OF THE LOAN AGREEMENT ARE HEREBY
EXPRESSLY INCORPORATED BY REFERENCE HEREIN.

          THIS NOTE SHALL BE GOVERNED BY, AND CONSTRUED IN ACCORDANCE WITH, THE
LAWS OF THE STATE OF GEORGIA WITHOUT GIVING EFFECT TO THE CONFLICT OF LAWS
PRINCIPLES THEREOF.









                     [SIGNATURES APPEAR ON FOLLOWING PAGE]


                                      -2-
<PAGE>   14
          IN WITNESS WHEREOF, the undersigned have executed this Note as of the
day and year first above written.




                                        RIDGEVIEW, INC.


[CORPORATE SEAL]


Attest:                                 By:   /s/ Hugh R. Gaither
                                           -----------------------------------
                                           Name:  Hugh R. Gaither
By:   /s/ Doug Yoder                              ----------------------------
   -----------------------------------     Title: President & CEO
   Name:  Doug Yoder                              ----------------------------
          ----------------------------
   Title: Assistant Secretary
          ----------------------------




                                        SENECA KNITTING MILLS
                                        CORPORATION

[CORPORATE SEAL]

Attest:                                 By:   /s/ Hugh R. Gaither
                                           -----------------------------------
                                           Name:  Hugh R. Gaither
                                                  ----------------------------
By:   /s/ Walter C. Bost, Jr.              Title: President & CEO
   -----------------------------------            ----------------------------
   Name:  Walter C. Bost, Jr.
          ----------------------------
   Title: Secretary
          ----------------------------


                                      -3-