A related party transaction refers to any transaction between the organization and a related party, which includes a disqualified person, a taxable entity, or a controlled entity. These transactions must be reported on Schedule R to ensure transparency and compliance with tax laws.