Annual meetings are typically held as membership meetings, which means that only a quorum of the members is required for the meeting to proceed.  A board member will typically chair and control the meeting and it is usually beneficial for the current directors to attend the meeting, but quorum of a membership meeting is tied to the number of members (not board members) being present.