Patent ID: 8514265

Claim:
A method, comprising: utilizing a processor of a videoconferencing system to perform operations including: receiving an audio stream from each of N remote endpoints that are participating in a videoconference, wherein N is greater than two, wherein the processor has access to M video decoders, wherein M is greater than one and smaller than N; analyzing the audio streams from the N remote endpoints to maintain a list of the M most recently voice-active endpoints among the N remote endpoints; receiving M video streams from the M most recently voice-active endpoints respectively; directing the decoding of the M video streams using respectively the M video decoders to generate M component images respectively; generating a composite image including at least the M component images; transmitting the composite image to one or more of the N remote endpoints; updating the list of M most recently voice-active endpoints to remove a least recently voice-active endpoint and to add a new endpoint in response to detecting current voice activity in the audio stream corresponding to the new endpoint; signaling the new endpoint to start transmitting a video stream; and reallocating a first of the M video decoders to the video stream transmitted from the new endpoint.