<PAGE>   1

                                                                      Exhibit 4e

                                 PROMISSORY NOTE

$20,000,000.00

                                  Dayton, Ohio
                                 April 29, 1998

For value received, the undersigned, HUFFY CORPORATION ("Borrower"), an Ohio
corporation, hereby promises to pay to the order of NATIONAL CITY BANK ("Bank"),
at its office at 6 North Main Street, Dayton, Ohio 45412, the principal sum of

                       TWENTY MILLION AND 00/1 00 DOLLARS

(or, if less, the unpaid principal balance shown on an attachment to this note
or on Bank's loan account records) together with interest, all as provided
below.

1. CROSS-REFERENCE -- Certain terms used in this note are defined in section 11.

2. NO COMMITMENT -- This Note evidences an arrangement whereby, for Borrower's
convenience, Borrower may, without having to execute and deliver a separate note
each time, obtain such loans (each a Subject Loan) as Borrower may from time to
time request and as Bank in its sole discretion may from time to time be willing
to make, subject in any case to the condition that (a) each Subject Loan shall
be in a minimum amount of five hundred thousand dollars ($500,000) and (b) the
aggregate unpaid principal balance of the Subject Loans shall not at any time
exceed the face amount of this Note. NOTWITHSTANDING ANY PROVISION OR INFERENCE
TO THE CONTRARY, BANK SHALL HAVE NO OBLIGATION TO EXTEND ANY CREDIT TO OR FOR
THE ACCOUNT OF BORROWER BY REASON OF THIS NOTE.

3. RECORDS; DISBURSEMENT OF PROCEEDS -- Whenever Borrower obtains a Subject
Loan, Bank shall (a) record the date and amount of each advance on Bank's loan
account records or on an attachment to this note, or both, each such entry to be
prima facie evidence of the information so entered, and (b) disburse the
proceeds (in the absence of Borrower's written instructions to the contrary) to
the credit of Borrower's general checking account with Bank. Bank shall not
transfer this note or seek judgment thereon or file proof of claim thereon
without first endorsing on this note the then actual principal balance of this
note.

4. SUBJECT LOANS -- Each Subject Loan shall bear interest at a fixed rate (the
Money Market Rate) and be payable on the last day of the applicable Contract
Period. The interest on each Subject Loan shall be payable on the last day of
the applicable Contract Period. After maturity (whether by lapse of time or
acceleration) each Subject Loan shall bear interest a rate per annum equal to
three percent (3%) per annum plus the Prime Rate from time to time in effect,
with each change in the Prime Rate automatically and immediately changing the
rate thereafter applicable to the Subject Loan, except that in no event shall
any Subject Loan bear interest after maturity at a lesser rate than that
actually applicable thereto immediately after maturity.


<PAGE>   2



5. PREPAYMENT -- Borrower may from time to time prepay the principal of the
Subject Loans in whole or in part subject to the following:

(a) Each prepayment shall be an amount equal to or greater than one million
dollars ($1,000,000) or, if less, the then balance due.

(b) Any prepayment of a Subject Loan made for any reason at any other time than
the last day of a Contract Period shall be subject to the payment of any
indemnity required by section 7.

(c) Concurrently with any prepayment of the principal of this note, Borrower
shall pay the unpaid interest accrued on the principal being prepaid.

6. FINANCIAL STATEMENTS -- Borrower will furnish to Bank the following:

(a) Within one hundred twenty (120) days from the end of Borrower's fiscal year,
balance sheets of Borrower for the fiscal year then ended and related statements
of revenue and expenses and changes in financial position for the fiscal year
then ended, all prepared in accordance with generally accepted accounting
principles applied on a consistent basis with those used in the preparation of
the financial statements of Borrower as of the prior report, and all audited or
reviewed by independent public accountants.

(b) Within forty-five (45) days from the end of each of Borrower's fiscal
quarters, Borrower's financial statements as of the end of the period, prepared
on a comparative basis with the prior year, in accordance with generally
accepted accounting principles, consistently applied, subject to any year end
adjustments and certified to by Borrower's chief financial officer.

(c) Prompt written notice of any final unappealable judgment or judgments
against Borrower in an amount exceeding one million dollars ($1,000,000) which
are not paid in full within thirty (30) days after the date of entry of the
judgment.

(d) Such other financial information as to Borrower as the Bank may, from time
to time, reasonably require.

7.1 INDEMNITY: FUNDING COSTS -- Borrower agrees to indemnify Bank against any
loss relating to its funding of any Subject Loan paid before its stated maturity
(whether a prepayment or a payment following any acceleration of maturity) and
to pay Bank, as liquidated damages for any such loss, a premium based upon the
principal amount of the Subject Loan prepaid and computed (on the basis of a
360-day year and actual days elapsed) at a rate per annum equal to the excess,
if any, of the applicable Money Market Rate hereunder over the Reinvestment
Rate, for the period from the date of prepayment to the stated maturity date on
which the principal was to be payable.

8. COSTS -- Any costs incurred by Bank in collecting the Subject Loans,
including reasonable attorneys fees, shall be paid by Borrower.



<PAGE>   3



9. LATE CHARGE; APPLICATION OF PAYMENTS -- If Borrower fails to pay any amount
due hereunder, or any fee in connection herewith, in full within ten (10) days
after its due date, Borrower will, in each case, incur and shall pay a late
charge equal to the greater of twenty dollars ($20.00) or five percent (5%) of
the unpaid amount. The payment of a late charge will not cure or constitute a
waiver of any Event of Default under this Agreement. Except as otherwise
provided in writing, payments will be applied first to accrued but unpaid
interest and fees, in that order, on an invoice by invoice basis in the order of
their respective due dates, until paid in full, then to late charges and then to
principal. In its discretion, Bank may, from time to time, unilaterally change
any provision for the application of payments by mailing a written notice to
Borrower of the change. The notice shall be mailed via registered or certified
mail, return receipt requested, to the address indicated herein or such other
address that Borrower may furnish in writing to an appropriate officer of Bank
and shall be mailed not less than fifteen (15) days prior to the effective date
of such change.

10. DEFAULT -- Upon the occurrence of any Event of Default, the holder of this
note may, in its sole discretion, by giving written notice to Borrower,
accelerate the maturity of the Subject Loans and thereupon the principal of and
accrued interest on the Subject Loans shall (if not already due) become
immediately due and payable without presentment or demand of any kind and
without any further or other notice of any kind. Upon the occurrence of any
Event of Default referred to in (e) or (f) of this section, the principal of and
accrued interest on the Subject Loans shall (if not already due) become
immediately due and payable (i.e., shall be accelerated) without notice and
without presentment or demand of any kind. The occurrence of any of the
following shall constitute an Event of Default hereunder: (a) Borrower's Bank
Debt or any part thereof shall not be paid in full promptly when due (whether by
lapse of time, acceleration of maturity or otherwise); (b) Borrower shall be
dissolved; (c) any representation or warranty made by Borrower in this note or
any Related Writing shall be false or erroneous in any material respect; (d)
Borrower shall fail or omit to perform or observe any agreement made by Borrower
in this note or in any Related Writing; (e) Borrower shall commence any
Insolvency Action of any kind or admit the material allegations of, or consent
to any relief requested in, any Insolvency Action of any kind commenced against
Borrower by its creditors or any thereof; (f) any creditor or creditors shall
commence against Borrower any Insolvency Action of any kind which shall remain
in effect (neither dismissed nor stayed) for sixty (60) consecutive days; or (g)
Borrower enters into any merger or consolidation in which it is not the
surviving entity or sells, leases, or otherwise disposes of all or substantially
all of its assets in any manner other than in the ordinary course of business.

11. DEFINITIONS -- In this note, Bank Debt means Debt payable to Bank, whether
initially payable to Bank or acquired by Bank by purchase, pledge or otherwise
and whether assigned or participated to or from Bank in whole or in part;

Banking Day means any day other than a Saturday or a Sunday or a public holiday
or other day on which banking institutions in Dayton, Ohio, are generally closed
and do not conduct a banking business;

Contract Period means a period of time which shall commence on a Banking Day and
end on a


<PAGE>   4



day at least one (1) day but not more than ninety (90) days thereafter,
provided, that (i) each such Contract Period shall be subject to Bank's assent
thereto and (ii) if any such Contract Period would otherwise end on a day that
is not a Banking Day, it shall end instead on the next succeeding Banking Day;

Debt means, collectively, all monetary liabilities, and any charges or expenses
incurred in connection therewith, now or hereafter owing by the Person or
Persons in question, including, without limitation, every such liability whether
owing by such Person or one (1) of such Persons alone or jointly, severally or
jointly and severally, whether owing absolutely or contingently, or directly or
indirectly, and whether created by loan, overdraft, guaranty or other contract
or by quasi-contract, tort, statute or other operation of law;

Event of Default is defined in section 10;

Insolvency Action means either (a) a pleading of any kind filed by the person,
corporation or entity (an insolvent) in question to seek relief from the
insolvent's creditors, or filed by the insolvent's creditors or any thereof to
seek relief of any kind against that insolvent, in any court or other tribunal
pursuant to any law (whether federal, state or other) relating generally to the
rights of creditors or the relief of debtors or both, or (b) any other action of
any kind commenced by an insolvent or the insolvent's creditors or any thereof
for the purpose of marshaling the insolvent's assets and liabilities for the
benefit of the insolvent's creditors; and Insolvency Action includes (without
limitation) a petition commencing a case pursuant to any chapter of the federal
bankruptcy code, any application for the appointment of a receiver, trustee,
liquidator or custodian for the insolvent or any substantial part of the
insolvent's assets, and any assignment by an insolvent for the general benefit
of the insolvent's creditors;

Money Market Rate means, with respect to a Subject Loan, the rate per annum as
determined by Bank, in its sole discretion, on the first day of the Contract
Period requested by Borrower for that Subject Loan, and then quoted by Bank to
Borrower as the rate which, if accepted by Borrower, will be applicable to that
Subject Loan during that Contract Period;

Person means a natural person or entity of any kind, including, without
limitation, any corporation, partnership, trust, governmental body, or any other
form or kind of entity;

Prime Rate means the fluctuating rate of interest which is publicly announced
from time to time by Bank at its principal place of business as being its "prime
rate" or "base rate" thereafter in effect, with each change in the Prime Rate
automatically, immediately and without notice changing the fluctuating interest
rate thereafter applicable hereunder, it being agreed that the Prime Rate is not
necessarily the lowest rate of interest then available from Bank on fluctuating
rate loans;

Reinvestment Rate means a rate of interest equal to the "bond equivalent yield"
for the most actively traded issues of U.S. Treasury Bills, U.S. Treasury Notes
or U.S. Treasury Bonds for a term similar to the period from (and including) the
date of prepayment to (but excluding) the due date of the principal of this Note
and in a principal amount comparable to the principal amount


<PAGE>   5



being prepaid, all as reasonably determined by Bank;

Related Writing means a writing of any form or substance signed by Borrower
(whether as principal or agent) or by any attorney, accountant or other
representative of Borrower and received by Bank in respect of Borrower's Bank
Debt or any part thereof, including, without limitation, any credit application,
credit agreement, reimbursement agreement, financial statement, promissory note,
guaranty, indenture, mortgage, security agreement, authorization, subordination
agreement, certificate, opinion or any similar writing, but shall not include
any commitment letter issued by Bank, without regard to whether Borrower or any
other Person signed or acknowledged receipt thereof;

Subject Loan means any loan made under this note;

the foregoing definitions shall be applicable to the respective plurals of the
terms defined above.

12. BUSINESS PURPOSE -- Borrower certifies to Bank that all funds disbursed
under this note will be used for business or commercial purposes.

13. INFORMATION -- Borrower hereby authorizes Bank to share all credit and
financial information relating to Borrower with Bank's parent company, and with
any subsidiary or affiliate company of Bank or of Bank's parent company.

14. WAIVERS; NOTICE -- No waiver, consent or other agreement shall be deemed to
have been made by Bank or Borrower or be binding upon Bank or Borrower unless
specifically granted in writing, which writing shall be strictly construed. Any
holder's delay or omission in the exercise of any right under this note shall
not operate as a waiver of that right or of any other right under this note. Any
notice to or demand upon Borrower shall be sufficiently made or given for all
purposes when sent by registered or certified mail, return receipt requested, to
the address hereinafter set forth but no other method of giving notice or making
demand is hereby precluded.

15. SEVERABILITY -- If any provision of this note is determined by a court of
competent jurisdiction to be invalid, illegal or unenforceable, that
determination shall not affect any other provision of this note, and each such
other provision shall be construed and enforced as if the invalid, illegal or
unenforceable provision were not contained herein.

16. ENTIRE AGREEMENT -- This note and the Related Writings set forth the entire
agreement between the parties regarding the transactions contemplated hereby,
and supersede all prior agreements, commitments, discussions, representations
and understandings, whether written or oral, and any and all contemporaneous
oral agreements, commitments, discussions, representations and understandings
between the parties relating to the subject matter hereof.



<PAGE>   6


17. AMENDMENT -- No amendment, modification or supplement to this note or any
Related Writing shall be binding unless executed in writing by all parties
thereto, and this provision shall not be subject to waiver by any party and
shall be strictly enforced.

18. OHIO LAW; INTEREST COMPUTATIONS -- This note shall be construed in
accordance with Ohio law. All interest shall accrue on the basis of a 360-day
year and actual days elapsed, but in no event at a higher rate than the maximum
rate (if any) permitted by law. Both principal and interest are payable at the
main office of Bank in lawful money of the United States.

Address: HUFFY CORPORATION
P.O. Box 1204
Dayton, Ohio 45401-1204
By: /s/Thomas A. Frederick
Printed Name: Thomas A. Frederick
Title: Vice President, Finance & C.F.O.