<PAGE>
                                                                    Exhibit 10.4

                               SIXTH AMENDMENT TO
                           THIRD AMENDED AND RESTATED
                           LOAN AND SECURITY AGREEMENT
                           ---------------------------

THIS SIXTH AMENDMENT TO THIRD AMENDED AND RESTATED LOAN AND SECURITY AGREEMENT
("Sixth Amendment") is made as of the 6 day of November, 2003 by and among PW
Eagle, Inc., a Minnesota corporation ("Borrower"), the lenders who are
signatories hereto ("Lenders"), and Fleet Capital Corporation, a Rhode Island
corporation ("FCC"), as agent for Lenders hereunder (FCC, in such capacity,
being "Agent").

                              W I T N E S S E T H:
                              - - - - - - - - - -

WHEREAS, Borrower, Agent and Lenders entered into a certain Third Amended and
Restated Loan and Security Agreement dated as of September 30, 2002 as amended
by a certain First Amendment to Third Amended and Restated Loan and Security
Agreement dated as of February 4, 2003 by and among Borrower, Lenders and Agent,
by a certain Second Amendment to Third Amended and Restated Loan and Security
Agreement dated as of May 30, 2003 by and among Borrower, Lenders and Agent, by
a certain Third Amendment to Third Amended and Restated Loan and Security
Agreement dated as of August 7, 2003 by and among Borrower, Lenders and Agent,
by a certain Fourth Amendment to Third Amended and Restated Loan and Security
Agreement dated as of September 15, 2003 by and among Borrower, Lenders and
Agent and by a certain Fifth Amendment to Third Amended and Restated Loan and
Security Agreement dated as of September 30, 2003 among Borrower, Lenders and
Agent (said Third Amended and Restated Loan and Security Agreement, as so
amended, is hereinafter referred to as the "Loan Agreement"); and
WHEREAS, Borrower desires to amend and modify certain provisions of the Loan
Agreement and, subject to the terms hereof, Agent and Lenders are willing to
agree to such amendments and modifications;
NOW THEREFORE, in consideration of the premises, the mutual covenants and
agreements herein contained, and any extension of credit heretofore, now or
hereafter made by Agent and Lenders to Borrower, the parties hereto hereby agree
as follows:
     1.   Definitions. All capitalized terms used herein without definition
shall have the meaning given to them in the Loan Agreement.

     2.   Financial Covenants. Upon the Sixth Amendment Effective Date, Exhibit
8.3 to the Loan Agreement is hereby deleted and Exhibit 8.3 attached hereto and
incorporated herein shall be inserted in its stead.

     3.   Applicable Margin. Borrower acknowledges that effective October 1,
2003, in accordance with the definition of such term, the Applicable Margin
shall be:

Base Rate Revolving Portion                    .50%
Base Rate Term Portion                         .75%
LIBOR Revolving Portion                       2.50%
LIBOR Term Portion                            2.75%

Thereafter the Applicable Margin shall be adjusted as provided in the definition
of such term.

<PAGE>

     4.   Amendment Fee. In order to induce Agent and Lenders to enter into this
Sixth Amendment, Borrower agrees to pay to Agent, for the ratable benefit of
Lenders, an amendment fee equal to $128,950. Said amendment fee shall be deemed
fully earned and non-refundable and shall be due and payable on the Sixth
Amendment Effective Date.

     5.   Conditions Precedent. This Sixth Amendment shall become effective upon
satisfaction of each of the following conditions precedent:

          (A)  Borrower, Agent and Lenders shall have executed and delivered to
     each other this Sixth Amendment;

          (B)  Borrower and the lenders under the Subordinated Note Documents
     shall have entered into an amendment to the Subordinated Note Documents in
     form and substance acceptable to Agent;

          (C)  Borrower and Lessor shall have entered in an amendment to the
     Sale and Leaseback Documents in form and substance acceptable to Agent; and

          (D)  Borrower shall have paid to Agent for the ratable benefit of
     Lenders the amendment fee referred to in Section 4 of this Sixth Amendment.

     The date on which all of the foregoing conditions precedent are satisfied
shall be called the "Sixth Amendment Effective Date."

     6.   Miscellaneous.

          (a)  This Sixth Amendment is limited as specified and shall not
     constitute an amendment, modification or waiver of any other provision of
     the Loan Agreement or any other Loan Document.

          (b)  This Sixth Amendment may be executed in any number of
               counterparts and by the different parties hereto on separate
               counterparts, each of which counterparts when executed and
               delivered shall be an original, but all of which shall together
               constitute one and the same instrument.

     7.   Continuing Effect. Except as otherwise specifically set out herein,
the provisions of the Loan Agreement shall remain in full force and effect.

                            (Signature Page Follows)

                                       2

<PAGE>

                      (Signature Page to Sixth Amendment to
             Third Amended and Restated Loan and Security Agreement)

IN WITNESS WHEREOF, this Sixth Amendment has been duly executed as of the day
and year specified at the beginning hereof.

                                        PW EAGLE, INC., ("Borrower")


                                        By:/s/ Scott Long
                                           -------------------------------------
                                           Name: Scott Long
                                                --------------------------------
                                           Title: CFO
                                                 -------------------------------

                                        FLEET CAPITAL CORPORATION, as
                                        Agent and as a Lender


                                        By: /s/ Brian Conole
                                           -------------------------------------
                                           Name: Brian Conole
                                                --------------------------------
                                           Title: Senior Vice President
                                                 -------------------------------


                                        THE CIT GROUP/BUSINESS CREDIT,
                                        INC., as Lender


                                        By: /s/ Anthony Alexander
                                           -------------------------------------
                                           Name: Anthony Alexander
                                                --------------------------------
                                           Title: Vice President
                                                 -------------------------------

                                       3

<PAGE>

                                                                        10.08.03
                                   EXHIBIT 8.3

                               FINANCIAL COVENANTS

Consolidated Net Income means, with respect to Borrower and its Subsidiaries for
any fiscal period, the net income (or loss) of Borrower and its Subsidiaries for
such period taken as a whole (determined in accordance with GAAP on a
consolidated basis), but excluding in any event: (a) any gains or losses on the
sale or other disposition of Investments or fixed or capital assets or from any
transaction classified as extraordinary under GAAP, any taxes on such excluded
gains and any tax deductions or credits on account of any such excluded losses;
(b) the proceeds of any life insurance policy; (c) net earnings and losses of
any business entity, substantially all the assets of which have been acquired in
any manner by Borrower, realized by such business entity prior to the date of
such acquisition; (d) net earnings and losses of any business entity which shall
have merged into Borrower earned or incurred prior to the date of such merger;
(e) net earnings of any business entity (other than a Consolidated Subsidiary)
in which Borrower has an ownership interest unless such net earnings shall have
been received by Borrower in the form of cash distributions; (f) earnings
resulting from a reappraisal, revaluation or write-up of assets; (g) any charge
to net earnings resulting from the amortization of the value of stock options
given to employees to the extent required by FASB 25; (h) any increase or
decrease of net income arising from a change in Borrower's accounting methods;
(i) any gains resulting from the forgiveness of Funded Debt or the retirement of
Funded Debt at a discount; (j) any gain arising from the acquisition of any
Securities of Borrower; and (k) any reversal of any contingency reserve, except
that provision for such contingency reserve shall have been made from income
arising during such period.

EBITDA With respect to any fiscal period, the sum of Borrower's Consolidated Net
Income plus amounts deducted in determining Consolidated Net Income in respect
of: (a) any provision for (or less any benefit from) income taxes whether
current or deferred; (b) amortization and depreciation expense; (c) Interest
Expense for such period; (d) prior to December 31, 1999, that portion of cost of
goods sold resulting from the write-up of Inventory in connection with the
Acquisition pursuant to APB 16; provided that the aggregate amount added to
EBITDA pursuant to this clause (d) shall not exceed $3,000,000; and (e) the
restructuring charge taken in the third fiscal quarter of fiscal year 2001 in
the amount of Two Million Five Hundred Thousand Dollars ($2,500,000.00).

Fixed Charge Coverage Ratio - With respect to any period of determination, the
ratio of (i) EBITDA of Borrower for such period minus income taxes paid in cash
and non-financed Capital Expenditures during such period to (ii) Fixed Charges.

Fixed Charges - For any period of determination, the sum of (a) scheduled
principal payments of Funded Debt (including the principal portion of scheduled
payments of Capital Lease Obligations), (b) Interest Expense paid in cash
included in the determination of Consolidated Net Income, and (c) dividends paid
on Borrower's capital stock.

                                       4

<PAGE>

Funded Debt - means: (i) Indebtedness arising from the lending of money by any
Person to Borrower, including, without limitation, the Obligations; (ii)
Indebtedness, whether or not in any such case arising from the lending by any
Person of money to Borrower (A) which is represented by notes payable or drafts
accepted that evidence extensions of credit, (B) which constitutes obligations
evidenced by bonds, debentures, notes or similar instruments, or (C) upon which
interest charges are customarily paid (other than accounts payable) or that was
issued or assumed as full or partial payment for Property; (iii) Indebtedness
that constitutes a Capitalized Lease Obligation; (iv) reimbursement obligations
with respect to letters of credit or guaranties of letters of credit and (v)
Indebtedness of Borrower for purposes of the Funded Debt to EBITDA Ratio for the
nine months ending September 30, 2002, EBITDA for such period shall be actual
EBITDA for such period multiplied by four-thirds (4/3s) for one most recently
ended twelve month period; provided, that, under any guaranty of obligations
that would constitute Funded Debt under clauses (i) through (iv) hereof if owed
directly by Borrower or any guaranty having the economic effect of guaranteeing
any of the obligations of any other Person. In computing the amount of Funded
Debt, the Subordinated Notes will be valued at full face value (less any
payments thereon) without giving effect to any original issue discount.

Funded Debt to EBITDA Ratio - With respect to any date, the ratio of (i) total
funded Funded Debt as of such date to (ii) EBITDA for the most recently ended
twelve month period; provided that for purposes of the Funded Debt to EBITDA
Ratio for the nine months ending September 30, 2002, EBITDA for such period
shall be actual EBITDA for such period multiplied by four-thirds (4/3s).

Interest Coverage Ratio - With respect to any period of determination, the ratio
of (i) EBITDA for such period to (ii) Interest Expense paid in cash for such
period, all as determined in accordance with GAAP.

Interest Expense - With respect to any fiscal period, the interest expense
incurred for such period excluding interest income as determined in accordance
with GAAP.

Investment - All investments in the property or assets of any person, in cash or
property, whether by way of advance, loan, extension of credit by Borrower or
any of its Subsidiaries (by way of guaranty or otherwise) or capital
contribution, or purchase of stock, bonds, notes, debentures or other securities
or any assets constituting the purchase of a business or line of business.

Net Worth - Book net worth of Borrower as determined in accordance with GAAP.
For purposes of this Exhibit 8.3, Net Worth shall include any unamortized value
assigned to the Warrants issued in connection with the Subordinated Notes which
value was calculated in accordance with GAAP and is contained in Borrower's
Consolidated Financial Statements.

                                       5

<PAGE>

Interest Coverage Ratio - Borrower shall not permit the Interest Coverage Ratio
as of the last date of the period set forth below to be less than the ratio set
forth opposite such period below:

--------------------------------------------------------------------------------
                          Period                                      Ratio
                          ------                                      -----
--------------------------------------------------------------------------------
For 9 months ending 9/30/2002                                       1.80 to 1
--------------------------------------------------------------------------------
For 12 months ending 12/31/2002                                     1.65 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 3/31/2003 and 6/30/2003             1.70 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 9/30/2003                            .40 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 12/31/2003                          1.70 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 3/31/2004 and each 6/30,            1.70 to 1
9/30, 12/31 and 3/31 thereafter
--------------------------------------------------------------------------------

Fixed Charge Coverage Ratio - Borrower shall not permit the Fixed Charge
Coverage Ratio as of the last date of the period set forth below to be less than
the ratio set forth opposite such period below:

--------------------------------------------------------------------------------
             Period                                                  Ratio
             ------                                                  -----
--------------------------------------------------------------------------------
For 9 months ending 9/30/2002                                     1.20 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 12/31/2002                        1.10 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 3/31/2003 and 6/30/2003           1.10 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 9/30/2003                          .15 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 12/31/03                           .45 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 3/31/2004 and each 6/30,          1.10 to 1
9/30, 12/31 and 3/31 thereafter
--------------------------------------------------------------------------------

                                       6

<PAGE>

Net Worth - Borrower shall achieve Net Worth as of each day set forth in the
schedule below of not less than the amount set forth opposite such period of the
following schedule:

--------------------------------------------------------------------------------
           Period                                             Amount
           ------                                             ------
--------------------------------------------------------------------------------
12/31/2002                                                  $26,500,000
--------------------------------------------------------------------------------
3/31/2003                                                   $24,500,000
--------------------------------------------------------------------------------
6/30/2003                                                   $27,500,000
--------------------------------------------------------------------------------
9/30/2003                                                   $18,850,000
--------------------------------------------------------------------------------
12/31/2003                                                  $29,000,000
--------------------------------------------------------------------------------
3/31/2004                                                   $27,000,000
--------------------------------------------------------------------------------
6/30/2004                                                   $30,000,000
--------------------------------------------------------------------------------
9/30/2004                                                   $33,000,000
--------------------------------------------------------------------------------
12/31/2004                                                  $31,500,000
--------------------------------------------------------------------------------
3/31/2005                                                   $29,500,000
--------------------------------------------------------------------------------
6/30/2005                                                   $32,500,000
--------------------------------------------------------------------------------

Funded Debt to EBITDA Ratio - Borrower shall not permit the Funded Debt to
EBITDA Ratio for any period set forth below to be greater than the ratio set
forth opposite such period below:

--------------------------------------------------------------------------------
              Period                                             Ratio
              ------                                             -----
--------------------------------------------------------------------------------
9 months ending 9/30/2002                                      4.50 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 12/31/2002                     4.50 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 3/31/2003                      5.00 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 6/30/2003                      5.00 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 9/30/2003                     28.00 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 12/31/2003                     4.50 to 1
--------------------------------------------------------------------------------
Trailing 12 month period ending 3/31/04                        4.50 to 1
and each 6/30, 9/30/ 12/31 and 3/31 thereafter
--------------------------------------------------------------------------------

                                       7