Abstract:
Techniques allow seamless transitioning of a mesh-based videoconference to a bridged videoconference. Criteria are defined for use in a triggering point function that may be evaluated to determine whether to make the transition. Escalation logic in an escalation manager may be used to manage the transition, allowing some participants to connect via a multipoint control unit, which is in turn a participant in the mesh-based videoconference, with the escalation manager relaying media streams from the mesh-connected participants. The mesh-connected participants may then drop their mesh connections without disruption to the videoconference. Escalation logic in the endpoints may be used as an alternate to an escalation manager unit.

Description:
TECHNICAL FIELD 
       [0001]    The present invention relates to the field of videoconferencing, and in particular to a technique for escalating an ongoing mesh videoconferencing session to a bridged videoconferencing session without interrupting the videoconference. 
       BACKGROUND ART 
       [0002]    In a multi-party conference call in meshed mode all the participants send their media to each other directly. If this meshed call is escalated to a bridged mode, due to legacy endpoints joining the conference or exceeding the maximum number of participants in meshed mode, then all the participants in the existing conference call are forced to join the conference call on an Audio/Video Bridge such as a multipoint control unit (MCU). This transition from a meshed call to a bridged call creates a disruption in the active conference. This escalation or transition sometimes takes a significant amount of time, producing a blackout period in the conference call. Thus, users may experience discontinuity in the video and audio streams of their conference call when escalation happens. 
         [0003]    Since network and device capabilities are changing rapidly, the criteria to decide when to convert a mesh call to a bridge call can be very dynamic. Currently the criteria are mainly the number of participants. With the increase in the network bandwidth, and the use of mobile endpoints and standards like WEBRTC (“Web Real-Time Communication”) there are other factors that can affect the user experience in a meshed-based conference call. This can create very bad experiences for users if their device is not capable of supporting the number of streams needed for the conference or the network conditions are not good. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0004]    The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an implementation of apparatus and methods consistent with the present invention and, together with the detailed description, serve to explain advantages and principles consistent with the invention. In the drawings, 
           [0005]      FIG. 1  is a flowchart illustrating a technique for triggering escalation of a mesh-based conference to a bridged conference according to one embodiment. 
           [0006]      FIG. 2  is a block diagram illustrating a mesh-based conference according to one embodiment. 
           [0007]      FIG. 3  is a block diagram illustrating another endpoint attempting to join the conference of  FIG. 2 . 
           [0008]      FIG. 4  is a block diagram illustrating use of an escalation manager according to one embodiment for managing the escalation of the mesh-based conference of  FIG. 2  to a bridged conference. 
           [0009]      FIG. 5  is a block diagram of the conference of  FIG. 4 , transitioned to a fully bridged conference according to one embodiment. 
           [0010]      FIG. 6  is a block diagram of a mesh-based conference in transition to a bridged conference according to one embodiment. 
           [0011]      FIG. 7  is a block diagram of the conference of  FIG. 6  after transitioning to a bridged conference. 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
       [0012]    In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without these specific details. In other instances, structure and devices are shown in block diagram form in order to avoid obscuring the invention. References to numbers without subscripts or suffixes are understood to reference all instance of subscripts and suffixes corresponding to the referenced number. Moreover, the language used in this disclosure has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter. Reference in the specification to “one embodiment” or to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least one embodiment of the invention, and multiple references to “one embodiment” or “an embodiment” should not be understood as necessarily all referring to the same embodiment. 
         [0013]    As used herein, the term “a computer system” can refer to a single computer or a plurality of computers working together to perform the function described as being performed on or by a computer system. 
         [0014]    In the following description, the conference call or conference session may be either audio-only, video-only, or a mixture of audio and video. In addition, although not illustrated in the Figures for clarity, data-only streams may be provided in the conference session. The terms “call” and “session” are used synonymously herein. 
         [0015]    The description below lists a variety of metrics to be considered to identify the triggering point for converting from a mesh conference to a bridged conference. A mechanism is proposed to collect the metrics and calculate a weighted index of the criteria so the triggering point can be derived. 
         [0016]    In a meshed based conference call, each participant has to establish a separate media connection with each other participant. This consumes lot of CPU cycles/processing power on device. It also consumes lot of network bandwidth. 
         [0017]    The following are some example criteria to be considered when deciding on an escalation to a bridge call. 
         [0018]    1. Number of participants in the meeting. (N) 
         [0019]    As more participants join the conference, the number of media streams maintained by each endpoint will increase. This may create a bad user experience when the number of streams approaches or exceeds the capacity of an endpoint. 
         [0020]    2. Device bandwidth limitation for the meshed user. (D1) 
         [0021]    In some scenarios, the endpoint capacity is sufficient for a mesh call, but network bandwidth or quality limitations may produce negative results. In such a scenario, handling multiple media connections in that network can be a problem. 
         [0022]    3. Device computational limitation for the meshed user. (D2) 
         [0023]    Mobile devices comes with a wide range of capacity. For example, a low end mobile device may not be capable of handling multiple media streams. 
         [0024]    4. Device power consumption for the meshed user. (D3) 
         [0025]    For endpoints that are mobile devices, the power which can consumed for computation may also be limited. 
         [0026]    5. Mobility of the device for the meshed user. (M) 
         [0027]    Where endpoints are mobile devices, network routing and network address translation traversals also can change during the call. These can trigger significant Session Description Protocol (SDP) renegotiation between participants, which can create bad experiences for some users. 
         [0028]    5. Bridge user joins (B) 
         [0029]    If a bridge user wants to the join a conference call that is currently running in a meshed based conference call. 
         [0030]    The triggering point may be considered a function of all the above criteria. These criteria are illustrative and by way of example only, and other criteria may be defined and used as desired. Although multiple criteria are described above, a single criterion, such as the number of participants, may be used if desired.
       Triggering point=F (N, D1, D2, D3, M, B)       
 
         [0032]    The function may be a simple weighted index or any other mathematical model which may be configured in the system. 
         [0033]    To implement such a system in one embodiment, a conference monitor may collect the metrics from the endpoint device using agents or in an agentless way. The conference monitor may run real-time analytics on the metrics data received from the endpoint devices and calculate the triggering point value F(N, D1, D2, D3, M, B). If the calculated triggering point value meets or exceeds a predetermined configurable threshold value, the conference monitor may trigger escalation to a bridge call. Other relationships between the calculated triggering point and the threshold value may be used as desired. 
         [0034]    In one embodiment, the function used to determine the triggering point and the threshold may be customizable. Different functions and thresholds may be assigned to different conferences as desired. For example, some functions may weight the various parameters, giving different weights to different parameters as desired. Other functions may omit one or more of the above parameters, and may include one or more other parameters not described above. Based on the criteria used and the weighting of the parameters, an optimal threshold for a triggering point may be identified for each conference session. 
         [0035]    In addition to defining a triggering point function that is more robust that merely counting the number of participants, the disclosed system provides a better user experience because the escalation of the meshed call to the bridged call is seamless, performing the escalation without breaking the existing conversation. 
         [0036]      FIG. 1  is a flowchart illustrating a technique  100  according to one embodiment NAT outlined above. In block  110 , each conference is assigned a triggering point function and a triggering point threshold. Different conferences may employ different functions and thresholds. In some embodiments, the function or threshold may be changed as desired during an active conference. In block  120 , metrics relevant to the triggering point function are obtained from the endpoints in the conference. Metrics regarding network capacity and quality are also obtained. The metrics may be collected with or without the use of agents, and some embodiments may use agents for collecting some metrics and not use agents for collecting other metrics. In block  130 , the triggering point function is evaluated, in this example applying metric data for the number of participants, device bandwidth limitations, device computational limitations, mobility of the devices, and bridge users. In block  140 , the triggering point function value is evaluated against the threshold value. If the threshold is passed (or, in some embodiments, met), the conference is triggered to escalate to a bridged conference in block  160 . If the triggering point value does not pass (or meet) the threshold value, then in block  150  a check is made to determine whether the conference has ended. If not, then the technique repeats, beginning with the collection of metrics in block  120 . If the conference has ended then the technique finishes. 
         [0037]    We now turn to a technique for performing the escalation that is triggered by the evaluation of a triggering function such as the triggering function described above or based on any other triggering technique. In the following, the triggering function for clarity is devolved to a simple function of the number of participants, and the threshold is a number of participants N. However, the following escalation technique is not limited to such a “number of participants” function. 
         [0038]    Consider a meshed-based conference call with a threshold of N participants where currently there are N−1 active participants. When a new participant joins the conference call, the call needs to be escalated to a bridge call. This transition may be handled by an escalation manager. 
         [0039]    When a new participant tries to join the meshed conference call as a participant, the escalation manager redirects the new participant to a bridge directly. The escalation manager itself then joins the meshed conference as the Nth participant on the mesh side of the escalation manager with the MCU or bridge on the other side of the escalation manager. Over time, the call may move the meshed participants from the mesh side to the bridge side without interruption, providing for a seamless transition to a bridge conference. 
         [0040]      FIG. 2  is a block diagram illustrating a mesh-based conference session  200  among three endpoints  210 A-C according to one embodiment. In this simple example, the threshold for triggering an escalation to a bridge conference is 4 participants and meeting the threshold triggers the escalation. Alternately, the threshold value may be 3 participants, and exceeding that threshold triggers the escalation. Although illustrated as laptop computers in  FIG. 1 , the endpoints  210  may be any type of computational device, including mobile devices. The endpoints  210  are connected to each other via network connections across one or multiple networks, which may include cellular networks. 
         [0041]    In  FIG. 3 , new participant  210 D attempts to join the conference session  200 , which would bring the conference to the threshold number of participants. In this embodiment, an escalation manager comes into play as illustrated in  FIGS. 4 and 5 . 
         [0042]    Endpoint  210 D contacts the escalation manager  420 , indicating a desire to join the conference with endpoints  210 A-C. The escalation manager  420  directs endpoint  210 D to connect to MCU  410 , and establishes a connection between the endpoint manager  420  and the MCU  410 . The escalation manager  420  also makes mesh connections with the other three endpoints in the mesh conference, so that the conference  200  is now a 4-way mesh conference. In some embodiments, the escalation manager  420  may perform other functionality for the conference  200 , in addition to the escalation functions. 
         [0043]    On the bridge side of the call, the escalation manager  420  sends to the MCU  410  the multiple media streams received or gathered from all the other N−1 conference participants  210 A-C in the meshed call. The MCU  410  sends a composed or stitched media stream of all the meshed conference participants media to the newly added participant  210 D. Although only a single stream between endpoint  210 D and MCU  410  is illustrated in  FIG. 4 , embodiments may send more than one stream as desired. In some embodiments, the stitched media stream may include segments from less than all of the other endpoints  210 A-C as desired. The MCU  410  also sends back to the escalation manager  420  a media stream from endpoint  210 D (the N+1st participant). The escalation manager  420  then forks the stream to all the meshed call participants  210 A-C. In one embodiment, the escalation manager  420  adds a specific token for each media stream sent to the MCU  410 , so that the MCU  410  considers the streams as media from single source, and indicating the MCU  410  should not send the stream from escalation manager  420  back in the stitched video stream to the endpoints  210 A-C. 
         [0044]    If yet another participant (in  FIG. 4 , endpoint  210 E) joins the conference  200 , that endpoint joins via the bridge  410  only. In one embodiment, the MCU  410  sends an additional media stream to the escalation manager  420  for each endpoint connected directly to the bridge  410 . In other embodiments, a single media stream is sent from the MCU  410  to the escalation manager, combining media streams from each of the participants connected directly to the bridge MCU  410 . 
         [0045]    The conference then continues as a mixed mesh-bridge conference. In one embodiment, the escalation manager  420  can decide if the entire meshed conference needs to move onto the MCU  410 , using a triggering technique similar to the triggering technique described above. 
         [0046]    In one embodiment, the escalation manager slowly stops the peer-to-peer media flow between meshed endpoints  210 A-C, while directing or requesting the MCU  410  to add additional media streams for each endpoint  210 A-C that drops mesh connections, as illustrated in  FIG. 5 . In such an embodiment, the escalation manager  420  may then function as a media relay MCU, relaying media streams to and from the MCU  410  as a proxy. 
         [0047]    In one embodiment, the escalation manager  420  can monitor the conference, listening for active speakers. The escalation manager  420  may trigger a sequence for endpoints corresponding to non-active speakers to drop their meshed connections. The escalation manager  420  also sends a notification to MCU  410 , to cause the MCU  410  to start sending a separate stream intended for the endpoint  210  that is no longer part of the mesh. By snapping only non-active speaker endpoints from mesh to bridge conference connections, the conference transition appears seamless to the participant at the endpoint  210  whose connection has switched from mesh to bridged. Over time, this may result in the situation illustrated in  FIG. 5 , in which all of the formerly meshed endpoints  210 A-C are connected as bridged endpoints via the escalation manager  420 . 
         [0048]    In another embodiment, illustrated in  FIGS. 6-7 , escalation manager  420  is not used to relay media streams and may be omitted. In one embodiment, instead of escalation logic in a separate escalation manager  420 , escalation logic is implemented in each of the endpoints  210 A-D. In another embodiment, an escalation manager  420  may be used, but serves to direct the endpoints  210  and MCU  410  to perform the escalation, without serving as a relay for streams. The conference starts out as a meshed conference as illustrated in  FIG. 2 . But in this embodiment, when endpoint  210 D attempts to join the conference, escalation logic in each of the endpoints causes the MCU  410  to join the mesh conference as the Nth participant and endpoint  210 D to join the conference as the N+1 st  endpoint directly on the MCU  410 , as illustrated in  FIG. 5 . 
         [0049]    Then, over time when each of endpoints  210 A-C is not an active speaker, that endpoint  210  drops its mesh connections to the other endpoints  210 , and communicates only through the MCU  410 , eventually resulting in the fully bridged conference of  FIG. 7 . 
         [0050]    As illustrated in  FIG. 6 , all the current mesh-based conference participants  210 A-C dial into the bridge MCU  410  while keeping their meshed call legs up and running. This ensures that the running conversation is not broken. 
         [0051]    In one embodiment, the call to join the MCU  410  may contain a specific flag or identifier which will enables the MCU  410  to consider all the media coming from the original N−1 participants, endpoints  210 A-C, as media coming from a special source that is treated logically as a single entity. In this embodiment, the media stream sent by the MCU  410  to the mesh-based participant endpoints  210 A-C does not contain any audio or video of the meshed endpoints  210 A-C, but only media from the participants that have joined the conference call directly on the MCU  410 . 
         [0052]    On the other hand, endpoint  210 D, which is hosted on the MCU  410 , receives media streams from the MCU  410  from all of the meshed conference participants  210 A-C and any other bridge-connected participant. 
         [0053]    Similar to the transition illustrated in  FIGS. 4-5 , as the conference progresses, at a given point of time there may be a subset m of the N participants that active speakers. Leaving the m active speakers untouched, escalation logic in the endpoints  210 A-C can remove the rest of the other N-m participants from the meshed call. In one embodiment, this is performed sequentially, one by one, by the following procedure. 
         [0054]    First, the existing peer-to-peer connection legs (with meshed users) are removed for an endpoint  210  that is to transition to fully-bridged mode. Next, a notification is sent to the MCU  410  to remove that participant from the single source context, so that the MCU sends the endpoint  210  media for all the other participants in the conference, not just the bridge-only participants. 
         [0055]    After applying the above steps, endpoints  210  without active speakers may transition smoothly to the MCU  410  by simply turning off their meshed connections and detaching from the common source context. 
         [0056]    In one embodiment, all the meshed users join the call on the MCU  410  with a special key (shared in Session Initiation Protocol or Session Description Protocol), that enables the MCU  410  to keep the Real-Time Protocol (RTP) media streams coming from meshed endpoints  210  into a single source. 
         [0057]    By keeping RTP into the same context/source, while stitching the media, the MCU  410  does not send media in the stitched stream back to the meshed participants. 
         [0058]    For example, using the example illustrated in  FIGS. 6-7 , endpoints  210 A-C are on the meshed call; thus, they are directly getting media streams from each other. When endpoint  210 D joins the call on the MCU  410 , escalation happens and endpoints  210 A-C send an SDP Invite to MCU  410  containing their audio video ports. In this Invite, endpoints  210 A-C share a common key, allowing the MCU  410  to consider the RTP media streams coming from these endpoints as a RTP stream coming from a single source. Therefore, the MCU  410  does not send the stitched media back to the endpoints  210 A-C, preventing echoing their streams back to the meshed endpoints  210 A-C that are still talking to each other over their mesh connections. 
         [0059]    New participant endpoint  210 D, however, gets the media from all the meshed endpoints  210 A-C and all the meshed endpoints  210 A-C get media from endpoint  210 D as a conference participant in the stitched video coming back to the endpoints  210 A-C from the MCU  410 . When any of the endpoints  210 A-C transitions from mesh to bridged connections, the media stream from that endpoint  210  no longer contains the special key, causing the MCU  410  to include media from that endpoint  210  in the stitched media stream. Once that happens, the endpoints  210  may simply drop their mesh connections. 
         [0060]    The escalation logic described above, whether implemented as escalation manager  420  or as escalation logic of an endpoint  210 , may be constructed and implemented in either software, hardware, firmware, or any combination thereof 
         [0061]    Program instructions may be used to cause a general-purpose or special-purpose processing system that is programmed with the instructions to perform the operations described herein. Alternatively, the operations may be performed by specific hardware components that contain hardwired logic for performing the operations, or by any combination of programmed computer components and custom hardware components. The methods described herein may be provided as a computer program product that may include a machine readable medium having stored thereon instructions that may be used to program a processing system or other electronic device to perform the methods. The term “machine readable medium” used herein shall include any medium that is capable of storing or encoding a sequence of instructions for execution by the machine and that cause the machine to perform any one of the methods described herein. The term “machine readable medium” shall accordingly include, but not be limited to, tangible, non-transitory memories such as solid-state memories, optical and magnetic disks. Furthermore, it is common in the art to speak of software, in one form or another (e.g., program, procedure, process, application, module, logic, and so on) as taking an action or causing a result. Such expressions are merely a shorthand way of stating that the execution of the software by a processing system causes the processor to perform an action or produce a result. 
         [0062]    It is to be understood that the above description is intended to be illustrative, and not restrictive. For example, the above-described embodiments may be used in combination with each other. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the invention therefore should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.