<PAGE>   1
                                                                    Exhibit 10.1

                               FOURTH AMENDMENT TO
                                CREDIT AGREEMENT

         THIS FOURTH AMENDMENT TO CREDIT AGREEMENT ("Fourth Amendment") dated as
of September 21, 1998 is by and among BALDWIN PIANO & ORGAN COMPANY, a Delaware
corporation, (hereinafter, together with its successors in title and assigns
called "Borrower" or "Baldwin"), THE FIFTH THIRD BANK, an Ohio banking
corporation, as Agent (in such capacity, the "Agent"), THE FIFTH THIRD BANK
("Fifth Third"), as a Lender, and NBD BANK, N.A., a national banking
association, ("NBD") as a Lender, (Fifth Third and NBD are hereinafter
collectively the "Lenders" and each individually a "Lender").


                              PRELIMINARY STATEMENT

         WHEREAS, Borrower, Agent and Lenders have entered into a Credit
Agreement dated as of October 16, 1997, as amended by a First Amendment dated as
of October 16, 1997, a Second Amendment dated as of April 27, 1998, and a Third
Amendment dated June 19, 1998 (the "Credit Agreement"); and

         WHEREAS, Borrower have requested Agent and Lenders to amend the Credit
Agreement as set forth herein; and

         WHEREAS, Borrower, Agent and Lenders now wish to amend the Credit
Agreement in accordance with the terms and provisions hereof;

         NOW, THEREFORE, the parties hereto agree to supplement and amend the
Credit Agreement upon such terms and conditions as follows:

         1. CAPITALIZED TERMS. All capitalized terms used herein shall have the
meanings assigned to them in the Credit Agreement unless the context hereof
requires otherwise. Any definitions as capitalized terms set forth herein shall
be deemed incorporated into the Credit Agreement as amended by this Fourth
Amendment.

         2. BORROWING BASE CERTIFICATE. During the period from the Fourth
Amendment Closing Date until December 15, 1998, Lenders hereby approve the 1998
Borrowing Base Certificate, attached as an Exhibit to this Fourth Amendment, as
the form of Borrowering Base Certificate in lieu of the form attached as Exhibit
C to the Credit Agreement.

         3. ELIGIBLE ACCOUNTS AND ELIGIBLE INVENTORY. Section 3.2(a)(ii) of the
Credit Agreement is hereby amended to in its entirety to read as follows:

                  "(ii) ELIGIBLE INVENTORY. On receipt of each Borrowing Base
         Certificate, Agent will credit Baldwin with the lesser of: (i)
         seventy-five percent (75%) of the 

<PAGE>   2

                                      -2-


         Value of Eligible Inventory which is, absent error or other
         discrepancy, listed in such Borrowing Base Certificate, and (ii) Thirty
         Five Million and 00/100 Dollars ($35,000,000.00) from the Fourth
         Amendment Closing Date through December 15, 1998, and Thirty Million
         Dollars and 00/100 Dollars ($30,000,000.00) after December 15, 1998.

         4. REAFFIRMATION OF COVENANTS, WARRANTIES AND REPRESENTATIONS. Borrower
hereby agrees and covenants that all representations and warranties in the
Credit Agreement, including without limitation all of those warranties and
representations set forth in Article 9, are true and accurate as of the date
hereof. Borrower further reaffirms all covenants in the Credit Agreement, and
reaffirms each of the affirmative covenants set forth in Section 10.1, the
negative covenants set forth in Section 10.2 and the financial covenants set
forth in Section 10.3 thereof, as if fully set forth herein, except to the
extent modified by this Fourth Amendment.

         5. CONDITIONS PRECEDENT TO CLOSING OF FOURTH AMENDMENT. On or prior to
the closing of the Fourth Amendment (hereinafter the "Fourth Amendment Closing
Date"), each of the following conditions precedent shall have been satisfied:

                  (a) LEGALITY OF TRANSACTIONS. No change in applicable law
         shall have occurred as a consequence of which it shall have become and
         continue to be unlawful (i) for Agent and each Lender to perform any of
         its agreements or obligations under any of the Loan Documents, or (ii)
         for Borrower to perform any of its agreements or obligations under any
         of the Loan Documents.

                  (b) PERFORMANCE, ETC. Except as set forth herein, Borrower
         shall have duly and properly performed, complied with and observed each
         of its covenants, agreements and obligations contained in each of the
         Loan Documents. Except as set forth herein, no event shall have
         occurred on or prior to the Fourth Amendment Closing Date, and no
         condition shall exist on the Fourth Amendment Closing Date, which
         constitutes a Default or an Event of Default.

                  (c) CHANGES; NONE ADVERSE. Since the date of the most recent
         balance sheets of Borrower delivered to Provident, no changes shall
         have occurred in the assets, liabilities, financial condition,
         business, operations or prospects of Borrower which, individually or in
         the aggregate, are material to Borrower, and Provident shall have
         completed such review of the status of all current and pending legal
         issues as Agent shall deem necessary or appropriate.

         6. MISCELLANEOUS. (a) Borrower shall reimburse Agent for all fees and
disbursements of legal counsel to Agent which shall have been incurred by Agent
in connection with the preparation, negotiation, review, execution and delivery
of this Fourth Amendment and the handling of any other matters incidental
hereto.

                  (b) All of the terms, conditions and provisions of the
Agreement not herein modified shall remain in full force and effect. In the
event a term, condition or provision of the 


<PAGE>   3



                                       -3-

Agreement conflicts with a term, condition or provision of this Fourth
Amendment, the latter shall govern.

                  (c) This Fourth Amendment shall be governed by and shall be
construed and interpreted in accordance with the laws of the State of Ohio.

                  (d) This Fourth Amendment shall be binding upon and shall
inure to the benefit of the parties hereto and their respective heirs,
successors and assigns.

                  (e) This Fourth Amendment may be executed in several
counterparts, each of which shall constitute an original, but all which together
shall constitute one and the same agreement.

      [Remainder of page intentionally left blank. Signature page follows.]



<PAGE>   4
                                     -4-


                  IN WITNESS WHEREOF, this Fourth Amendment has been duly
executed and delivered by or on behalf of each of the parties as of the day and
in the year first above written.


                                        BALDWIN PIANO & ORGAN 
                                        COMPANY, Borrower


                                        By:   /s/ Perry H. Schwartz
                                            --------------------------------
                                        Name:    Perry H. Schwartz
                                             -------------------------------
                                        Title:   EVP and CFO
                                              ------------------------------


                                        THE FIFTH THIRD BANK, Agent


                                        By:   /s/ Robert C. Ries
                                            --------------------------------
                                        Name:    Robert C. Ries
                                             -------------------------------
                                        Title:   Vice President
                                              ------------------------------



                                        THE FIFTH THIRD BANK, Lender


                                        By:   /s/ Robert C. Ries
                                            --------------------------------
                                        Name:    Robert C. Ries
                                             -------------------------------
                                        Title:   Vice President
                                              ------------------------------



                                        NBD BANK, N. A., Lender


                                        By:   /s/ Edward C. Hathaway
                                            --------------------------------
                                        Name:    Edward C. Hathaway
                                             -------------------------------
                                        Title:   First Vice President
                                              ------------------------------

<PAGE>   5


                                      -5-



                     FORM OF 1998 BORROWING BASE CERTIFICATE
                     ---------------------------------------
<TABLE>
<CAPTION>

BALDWIN PIANO & ORGAN COMPANY       Report #____________               Today's date_____
                                                                           Accounts Reported as of  ___
                                                                           Inventory Reported as of ___
<S>                                                                   <C>                     <C>

--------------------------------------------------------------------------------------------------------------------
I.     ELIGIBLE ACCOUNT AVAILABILITY
--------------------------------------------------------------------------------------------------------------------
         A.   ELIGIBLE ACCOUNT AVAILABILITY
1.            Accounts                                                                        $______________
              LESS:
2.            Due greater than 60 Days from date of sale               _____________
3.            Contract Electronics unpaid greater than 90 Days
              and others greater than 120 days
4.            Entire Account, if 50% or more unpaid greater than 
              90 Days                                                  _____________
5.            Affiliate Accounts                                       _____________
6.            Consignment Receivables greater than 120 Days            _____________
7.            Conditional                                              _____________
8.            Non U.S./Canada                                          _____________
9.            Non Reps & Warr. (per sec. 9.18 of Credit Agreement)     _____________
10.           Demonstration/Loan                                       _____________
11.           Progress/Barter/Contra                                   _____________
12.           Personal/family/household                                _____________
13.           Agent Designated Accounts                                _____________
14.           TOTAL INELIGIBLE  (sum lines 2 through 13)                                      (______________)
15.           SUBTOTAL                                                                        
16.           LESS: ELIGIBLE ACCOUNT NET AVAILABILITY                                         (______________)
          (15% times  Line 15)                                                                       
17.           TOTAL ELIGIBLE ACCOUNT                                                          $
          AVAILABILITY                                                                         ==============
         (Line 15, Less Line 16)
18.           TOTAL CREDIT FOR ELIGIBLE ACCOUNTS                                                               $_______________
         (Lesser of Line 17,  or $15,000,000)


--------------------------------------------------------------------------------------------------------------------
II.    ELIGIBLE INVENTORY AVAILABILITY
--------------------------------------------------------------------------------------------------------------------
              A.ELIGIBLE INVENTORY AVAILABILITY
19.           INVENTORY (Wholesale Dealer Cost)                                                ______________
              LESS:
20.           Work in Progress                                        (______________)
21.           Spare Parts/Shipping and Packaging Materials            (______________)
22.           Defective Inventory                                     (______________)
23.           Returned or repossessed                                 (______________)
24.           Non U.S./Canada                                         (______________)
25.           TOTAL INELIGIBLE INVENTORY:                                                     (______________)
                (Sum of Lines 20 through 24)                                                      
26.           SUBTOTAL  
27.           LESS: NET ELIGIBLE INVENTORY                                            
                (25% times Line 26)                                                              (______________)
28.           TOTAL ELIGIBLE INVENTORY AVAILABILITY 
              (Line 26 minus Line 27)
29.           TOTAL CREDIT FOR ELIGIBLE INVENTORY 
              (The Lesser of Line 28, or $35,000,000)
                                                                                                               $_______________
--------------------------------------------------------------------------------------------------------------------
III.   ELIGIBLE RAW MATERIALS AVAILABILITY
--------------------------------------------------------------------------------------------------------------------
              The Sum of:
30.           Value of Raw Materials/electronic (Baldwin's Cost)
31.           Times 10%                                                X .10 = $ ___________
32.           PLUS: The Value of all other Raw Materials
                (Baldwin's Cost)
33.           Times 50%                                                X .50 = + ___________
34.             TOTAL
</TABLE>


<PAGE>   6

<TABLE>
<CAPTION>
                                      -6-


                                                                                                   
35.           Raw Materials Availability                                                           _________________
                (Lesser of Line 34 or $5,000,000)
                                                                                                   

====================================================================================================================
36.  TOTAL BORROWING BASE AVAILABILITY                                        
     (Sum of lines 18, 29 and 35)                                                                 $_________________
====================================================================================================================

<S>            <C>                                                                                 <C>
37.           Lesser of Line 36 and 40,000,000.00 (or $44,000,000.00                               _________________
              as determined by the Credit Agreement)
                                                                                                   

                                                                                                   
38.           LESS:  Outstanding Standby Letters of Credit                                         (_______________)
                                                                                                   
                                                                                                   
                                                                                                   

                                                                                                   
39.           LESS:  Outstanding Revolving Credit Balance                                          (_______________)
                                                                                                  
                                                                                                  
                                                                                                  

====================================================================================================================
TOTAL AVAILABILITY TO BORROWERS 
(Sum of Lines 37, 38 and 39)                                                                      $_________________
====================================================================================================================

     We certify that the foregoing report is true and correct.


                                                                                       BALDWIN PIANO & ORGAN COMPANY


                                                                                       -----------------------------
                                                                                       Name:
                                                                                            ------------------------
                                                                                       Title:
                                                                                             -----------------------

</TABLE>