<PAGE>
                                                                    Exhibit 10.4

(BANK ONE LOGO)

                                                             LINE OF CREDIT NOTE

                                                                  $15,000,000.00
DUE: MAY 31, 2006                                            DATE: JUNE 15, 2005

PROMISE TO PAY. On or before May 31, 2006, for value received, Blue Valley Ban
Corp. (the "Borrower") promises to pay to JPMorgan Chase Bank, N.A., whose
address is 120 S. LaSalle, Chicago, IL 60603 (the "Bank") or order, in lawful
money of the United States of America, the sum of Fifteen Million and 00/100
Dollars ($15,000,000.00) or such lesser sum as is indicated on Bank records,
plus interest computed on the basis of the actual number of days elapsed in a
year of 360 days at the rate of 1.63% per annum above the Federal Funds Rate
(the "Note Rate"), and at the rate of 3.00% per annum above the Note Rate, at
the Bank's option, upon the occurrence of any default under this Note, whether
or not the Bank elects to accelerate the maturity of this Note, from the date
such increased rate is imposed by the Bank. In this Note, "Federal Funds Rate"
means the rate per annum equal to the consensus (or if no consensus exists, the
arithmetic average) of the rates at which reserves are offered by first-class
banks to other first-class banks (at approximately 8:00 a.m. CST time) on such
day (or if such day is not a Business Day, on the immediately preceding Business
Day) on overnight Federal funds transactions with members of the Federal Reserve
system arranged by brokers, received by the Bank from three Federal Funds
brokers of recognized standing selected by the Bank in its discretion, changing
when and as the Federal Funds Rate changes. The Bank's calculation of the
Federal Funds Rate shall be final, conclusive and binding on the Borrower in the
absence of manifest error.

In no event shall the interest rate exceed the maximum rate allowed by law. Any
interest payment that would for any reason be unlawful under applicable law
shall be applied to principal.

Interest will be computed on unpaid principal balance from the date of each
borrowing.

Accrued interest shall be due and payable on the last day of each March, June,
September and December, commencing June 30th, 2005. Principal shall be due and
payable at maturity.

The Borrower will pay, without setoff, deduction, or counterclaim, the Bank at
the Bank's address above or at such other place as the Bank may designate in
writing. If any payment of principal or interest on this Note shall become due
on a day that is not a Business Day, the payment will be made on the next
succeeding Business Day. The term "Business Day" in this Note means a day other
than a Saturday, Sunday or any other day on which national banking associations
are authorized to be closed. Payments shall be allocated among principal,
interest and fees at the discretion of the Bank unless otherwise agreed or
required by applicable law. Acceptance by the Bank of any payment that is less
than the payment due at that time shall not constitute a waiver of the Bank's
right to receive payment in full at that time or any other time.

AUTHORIZATION FOR DIRECT PAYMENTS (ACH DEBITS). To effectuate any payment due
under this Note, the Borrower hereby authorizes the Bank to initiate debit
entries to Account Number ________________________ at the Bank and to debit the
same to such account. This authorization to initiate debit entries shall remain
in full force and effect until the Bank has received written notification of its
termination in such time and in such manner as to afford the Bank a reasonable
opportunity to act on it. The Borrower represents that the Borrower is and will
be the owner of all funds in such account. The Borrower acknowledges (1) that
such debit entries may cause an overdraft of such account which may result in
the Bank's refusal to honor items drawn on such account until adequate deposits
are made to such account; (2) that the Bank is under no duty or obligation to
initiate any debit entry for any purpose; and (3) that if a debit is not made
because the above-referenced account does not have a sufficient available
balance, or otherwise, the payment may be late or past due.

LATE FEE. If any payment is not received by the Bank within ten (10) days after
its due date, the Bank may assess and the Borrower agrees to pay a late fee
equal to the greater of: (a) five percent (5.00%) of the past due amount or (b)
Twenty Five and 00/100 Dollars ($25.00), up to the maximum amount of One
Thousand Five Hundred and 00/100 Dollars ($1,500.00) per late charge.

CREDIT FACILITY. The Bank has approved a credit facility to the Borrower in a
principal amount not to exceed the face amount of this Note. The credit facility
is in the form of advances made from time to time by the Bank to the Borrower.
This Note evidences the Borrower's obligation to repay those advances. The
aggregate principal amount of debt evidenced by this Note is the amount
reflected from time to time in the records of the Bank. Until the earliest of
maturity, the occurrence of any default, or the occurrence of any event that
would constitute a default but for the giving of notice or the lapse of time or
both until the end of any grace or cure period, the Borrower may borrow, pay
down and reborrow under this Note subject to the terms of the Related Documents.

MISCELLANEOUS. This Note binds the Borrower and its successors, and benefits the
Bank, its successors and assigns. Any reference to the Bank includes any holder
of this Note. This Note is issued pursuant and entitled to the benefits of that
certain Credit Agreement by and

<PAGE>

between the Borrower and the Bank, dated June 15, 2005, and all replacements
thereof (the "Credit Agreement") to which reference is hereby made for a more
complete statement of the terms and conditions under which the loan evidenced
hereby is made and is to be repaid. The terms and provisions of the Credit
Agreement are hereby incorporated and made a part hereof by this reference
thereto with the same force and effect as if set forth at length herein. No
reference to the Credit Agreement and no provisions of this Note or the Credit
Agreement shall alter or impair the absolute and unconditional obligation of the
Borrower to pay the principal and interest on this Note as herein prescribed.
Capitalized terms not otherwise defined herein shall have the meanings assigned
to such terms in the Credit Agreement.

                                        BORROWER:

Address: 11935 Riley Street             Blue Valley Ban Corp.
         Overland Park, KS 66213


                                        By:
                                            ------------------------------------

                                            ------------------------------------
                                            Printed Name                   Title

                                        Date Signed:
                                                     ------------------


                                        2