<PAGE>   1
                                                                   EXHIBIT 10.12

                      SECOND AMENDMENT TO CREDIT AGREEMENT


Harris Trust and Savings Bank
Chicago, Illinois

Ladies and Gentlemen:

         Reference is hereby made to that certain Credit Agreement dated as of
January 30, 1998 (the "Credit Agreement"), between the undersigned, Midwest Banc
Holdings, Inc. and Midwest One Mortgage Services, Inc., as "Borrowers", and
Harris Trust and Savings Bank, as "Bank." All capitalized terms used herein
without definition shall have the same meanings herein as such terms have in the
Credit Agreement.

         The Borrowers have requested that the Bank modify Section 8.10 (Minimum
Consolidated Tangible Net Worth) and eliminate Section 8.16 (Dividends and
Certain other Restricted Payments), and the Bank is willing to do so under the
terms and conditions set forth in this agreement (herein, the "Amendment").

Section 1.   Amendments.

         Subject to the satisfaction of the conditions precedent set forth in
Section 2 below, the Credit Agreement shall be and hereby is amended as follows:

                  1.1      Section 8.10 of the Credit Agreement shall be amended
         and restated in its entirety to read as follows:

                           Section 8.10. Minimum Consolidated Tangible Net
                  Worth. The Company shall at all times maintain Consolidated
                  Tangible Net Worth of not less than $65,000,000.

                  1.2      Section 8.16 of the Credit Agreement shall be amended
         and restated in its entirety to read as follows:

                           Section 8.16.  Intentionally deleted.

Section 2.   Conditions Precedent.

         The effectiveness of this Amendment is subject to the satisfaction of
all of the following conditions precedent:

                  (a) The Borrowers and the Bank shall have executed and
         delivered this Amendment.


<PAGE>   2


                  (b) Legal matters incident to the execution and delivery of
         this Amendment shall be satisfactory to the Bank and its counsel.

Section 3.   Representations.

         In order to induce the Bank to execute and deliver this Amendment, the
Borrowers hereby represent to the Bank that as of the date hereof the
representations and warranties set forth in Section 6 of the Credit Agreement
are and shall be and remain true and correct (except that the representations
contained in Section 6.5 shall be deemed to refer to the most recent
consolidated financial statements of the Company delivered to the Bank) and the
Borrowers are in compliance with the terms and conditions of the Credit
Agreement and no Default or Event of Default has occurred and is continuing
under the Credit Agreement or shall result after giving effect to this
Amendment.

Section 4.   Miscellaneous.

         (a) The Company has heretofore executed and delivered to the Bank
certain Collateral Documents and the Company hereby acknowledges and agrees
that, notwithstanding the execution and delivery of this Amendment, the
Collateral Documents remain in full force and effect and the rights and remedies
of the Bank thereunder, the obligations of the Company thereunder and the liens
and security interests created and provided for thereunder remain in full force
and effect and shall not be affected, impaired or discharged hereby. Nothing
herein contained shall in any manner affect or impair the priority of the liens
and security interests created and provided for by the Collateral Documents as
to the indebtedness which should be secured thereby prior to giving effect to
this Amendment.

         (b) Except as specifically amended herein, the Credit Agreement shall
continue in full force and effect in accordance with its original terms.
Reference to this specific Amendment need not be made in the Credit Agreement,
the Notes, or any other instrument or document executed in connection therewith,
or in any certificate, letter or communication issued or made pursuant to or
with respect to the Credit Agreement, any reference in any of such items to the
Credit Agreement being sufficient to refer to the Credit Agreement as amended
hereby.

         (c) The Borrowers agree to pay on demand all costs and expenses of or
incurred by the Bank in connection with the negotiation, preparation, execution
and delivery of this Amendment, including the fees and expenses of counsel for
the Bank.

         (d) This Amendment may be executed in any number of counterparts, and
by the different parties on different counterpart signature pages, all of which
taken together shall constitute one and the same agreement. Any of the parties
hereto may execute this Amendment by signing any such counterpart and each such
counterparts shall for all purposes be deemed to be an original. This Amendment
shall be governed by the internal laws of the State of Illinois.

                           [Signature Page to Follow]


                                       2
<PAGE>   3


         This Second Amendment to Credit Agreement is dated as of January 7,
1999.


                                        MIDWEST BANC HOLDINGS, INC.

                                        By:  /s/ Robert L. Woods
                                             ------------------------------
                                                 Name:  Robert L. Woods
                                                 Title:  President


                                        MIDWEST ONE MORTGAGE SERVICES, INC.

                                        By:  /s/ Daniel R. Kadolph
                                             ------------------------------
                                                 Name:  Daniel R. Kadolph
                                                 Title:  Treasurer


         Accepted and agreed to in Chicago, Illinois as of the date and year
last above written.


                                        HARRIS TRUST AND SAVINGS BANK

                                        By:  /s/ Michael S. Cameli
                                             ------------------------------
                                                 Name:  Michael S. Cameli
                                                 Title:  Vice President



                                       3