Multipoint conference scalability for co-located participants

A terminal device associated with a participant of the multipoint session can be identified. The multipoint session can be associated with a video and an audio stream. The terminal device can be a computing device comprising of an audio and video input, and a video output. The multipoint session can be a conference associated with a telephony session. The location of the terminal device and a conferencing device can be determined. The conferencing device can be associated with an audio input and output. If the terminal device is proximate to the conferencing device, a terminal device audio stream can be terminated and a conference device audio stream can be received. A video stream and an audio stream can be simultaneously presented upon the audio and video output of the conferencing device and terminal device. The video and the audio stream can be a synchronized stream of the session.

BACKGROUND

The present invention relates to the field of and multipoint conferencing, more particularly, to improving multipoint conference scalability for co-located participants.

Multipoint conferencing, an invaluable collaboration tool, enables multiple remote persons to participate in a collaboration session. Typically multipoint conferencing takes the form of videoconferencing, which includes a visual component and audio component. The visual component often includes a camera for capturing video and a display for presenting the captured video. The audio component typically includes a microphone to record audio (e.g., speech) and a speaker to play the recorded audio. Many times, a multipoint controller unit (MCU) is utilized to allow multiple persons in one location (e.g., boardroom) to communicate using one centralized terminal device (e.g., conference phone).

BRIEF SUMMARY

One aspect of the present invention can include a method, a computer program product, a system, and an apparatus for enhancing multipoint conferencing. In this aspect, a terminal device associated with a participant of the multipoint session can be identified. The multipoint session can be associated with a video and an audio stream. The terminal device can be a computing device comprising of an audio and video input, and a video output. The multipoint session can be a conference associated with a telephony session. The location of the terminal device and a conferencing device can be determined. The conferencing device can be associated with an audio input and output. If the terminal device is proximate to the conferencing device, a terminal device audio stream can be terminated and a conference device audio stream can be received. A video stream and an audio stream can be simultaneously presented upon the audio and video output of the conferencing device and terminal device. The video and the audio stream can be a synchronized stream of the session.

Another aspect of the present invention can include a system for enhancing multipoint conferencing comprising a presence engine, a bridge manager, and a mapping structure. The presence engine can determine the location of a terminal device associated with a multipoint session. The terminal device can be associated with an audio stream and a video stream. The multipoint session can be a conference associated with a telephony session. The bridge manager can add and remove a conferencing device from the session, where the conferencing device is associated with an audio stream. The mapping structure can link a conferencing device with at least one of a unique identifier and a location.

DETAILED DESCRIPTION

If a person is utilizing a personal device (e.g., laptop) to participate in a multipoint conference session within the same location (e.g., co-located) as the terminal device, problems can arise. One common problem is the degradation of audio quality from audio feedback. For example, audio output from one person's laptop speaker can feed into another person's laptop microphone and cause feedback or echo for all users participating in the conference. As such, the person must either terminate the use of the personal device or change location. In many instances, these solutions can be detrimental to the session and do not leverage existing infrastructure to allow flexible session configurations to exist.

The present disclosure is a solution for improving multipoint conference scalability for co-located participants. In the solution, a multipoint session can utilize a centralized conferencing device to merge audio channels for each terminal device (e.g. laptop) which is proximate to the conferencing device. That is, a proximate conferencing device (e.g., conference phone) can be used to convey audio while the terminal device can be used to convey video. As such, the disclosure can reduce the total audio channels for the session, improving scalability and reducing resource utilization. In one embodiment, the solution can be a component of a Voice over Internet Protocol architecture.

In one embodiment, the solution can be utilized with a session which implements video switching capabilities (e.g., active speaker presentation). In the embodiment, audio received from a proximate conferencing device can be analyzed to determine an active speaker utilizing proprietary and/or traditional audio stream selection.

FIG. 1is a flowchart diagram illustrating a method100for improving multipoint conferencing scalability for co-located participants in accordance with an embodiment of the inventive arrangements disclosed herein. In method100, a conferencing device (e.g., conference phone) can be utilized to reduce the quantity of audio streams of a multipoint session for co-located devices. As used herein, co-located can refer to the presence of a terminal device (e.g., laptop) within proximity of a conferencing device (e.g., conference phone) associated with a multipoint session. Multipoint session can be a semi-permanent interactive information exchange between three or more communicating devices. For example, the multipoint session can be a conference call between four participants, each utilizing a laptop to interact with each other. The multipoint session can include an audio component, a video component, and the like. The multipoint session can be a telephone conference, a videoconference, a Web conference, and the like.

As used herein multipoint session can be associated with Voice over Internet Protocol (VoIP) technologies. VoIP can be one or more methodologies for enacting communications and multimedia sessions over Internet Protocol. VoIP can include, but is not limited to, communication protocols, transmission technologies, and the like. Protocols can include, but is not limited to, Session Initiated Protocol, H.323, IP Multimedia Subsystem (IMS), Real-time Transport Protocol (RTP), and Session Description Protocol (SDP).

In one instance, the conferencing device can be a telephony device able to convey audio during a multipoint session. In the instance, an audio stream from the telephony device can be utilized simultaneously with video from the terminal device.

In step105, a multipoint session can be established. The session can be initiated by a participant interacting with an application such as a videoconferencing application executing on a terminal device. In step110, terminal devices associated with participants can be added to the session. The terminal devices can be identified based one or more unique identifiers. Unique identifiers can include, but is not limited to, telephone number, Internet Protocol (IP) address, Media Access Control (MAC) address, and the like. For instance, a contact list linking participants to a terminal device can be maintained enabling rapid management of participants and/or terminal devices. In one instance, terminal devices can be introduced into the session via a participant list associated with a client (e.g., terminal device) and/or an invitee list associated with a server (e.g., server230).

In step115, a terminal device can be selected. The terminal device can be selected based on one or more criteria, including, but not limited to, device identifier, participant information, and the like. In step120, if the terminal device location is different from a historic location (e.g., during call initiation), the method can proceed to step125, else return to step115. The location of the terminal device can be determined utilizing one or more location-based technologies, including, but not limited to, Global Positioning System (GPS), Assisted GPS (A-GPS), BLUETOOTH, Radio Frequency Identification (RFID), and the like. In one instance, terminal device location can be obtained through polling the device for a current location. In another instance, terminal device location can be conveyed by the terminal device upon determination of location change.

In step125, if a proximate conferencing device is identified, the method can proceed to step130, else return to step120. Step120,125can be repeated until a proximate conferencing device (e.g., conference phone) is identified. The proximity of the terminal device to a conferencing device can be determined based on one or more configurable settings. In one instance, the proximity of the terminal device to the conferencing device can be determined based on a physical distance. In the instance, one or more threshold values can be utilized to establish proximity. For instance, a conferencing device can be in proximity if the terminal device is less than four feet away. In one embodiment, traditional proxemics values (e.g., personal distance, social distance) can be utilized to determine threshold.

In step130, the identified proximate conferencing device can be activated if the conferencing device is not in use. Activation can include one or more actions permitting the inclusion of the conferencing device to the multipoint session. In one instance, the conferencing device identifier (e.g., alias, phone number) can be automatically determined based on the terminal device location. In the instance, the conferencing device identifier can be used to automatically associate the conferencing device to the multipoint session. For instance, a mapping can be utilized to determine the phone number of the identified proximate conferencing device which can be used to dial the conferencing device into the session. In another instance, the conferencing device can be manually activated and joined to the multipoint session via one or more participant actions.

In step135, the audio stream associated with the terminal device can be terminated. That is, video for a participant can be captured from the terminal device and audio for the participant can be recorded from the conferencing device. In one instance, multipoint session can support video switching which can be visually indicating a participant of interest (e.g., active speaker). In the instance, video switching can be maintained while simultaneously using terminal device for video input and conferencing device for audio input.

In step140, if there are more terminal devices associated with the session, the method can return to step115, else continue to step145. Step115-140can be continuously performed during the multipoint session, permitting dynamic resource allocation during the session. In step145, the session can be terminated and the method can end.

Drawings presented herein are for illustrative purposes only and should not be construed to limit the invention in any regard. It should be appreciated that method100can be performed in real-time or near real-time during a multipoint session. It should be appreciated that steps115-140can be performed at any time during a multipoint session. In one instance, steps115-140can be performed at commencement of the session. In another instance, steps115-140can be responsive to a device joining the session after the session has commenced.

FIG. 2is a schematic diagram illustrating a system200for improving multipoint session scalability for co-located participants in accordance with an embodiment of the inventive arrangements disclosed herein. System200can be present in the context of method100. In system200, terminal device210can be used concurrently with conferencing device250to permit multipoint session scalability to be achieved. System200components can be communicatively linked via network270. In one embodiment, system200can conform to a conventional Voice over Internet Protocol (VOIP) capable system.

As used herein, session236can be a multipoint session associated with computing device210,230,250. It should be appreciated that computing device210can include multiple computing devices operated by participants (e.g., human agents) of the session236. For instance, four participants can utilize personal laptops (e.g., video) and a conference phone (e.g., audio) while occupying a conference room during a multipoint session. Session236can include, but is not limited to, session information237, session history, session configuration, and the like.

Video262can be a digitally encoded artifact of input received from component video input215. Video262characteristics can include, but is not limited to, frames per second, interlacing, resolution, aspect ratio, color space, compression, bit rate, and the like. Video262can conform to one or more traditional and/or proprietary encoding formats, including, but not limited to, H.264, Motion Picture Expert Group 4 (MPEG-4), H.261, H.263, and the like.

Audio260can be a digitally encoded artifact of input received from component audio input252. Audio260characteristics can include, but is not limited to, sample rate, dynamic range, compression, and the like. Audio260can conform to one or more traditional and/or proprietary formats including, but not limited to, MPEG-4, Adaptive Multi-Rate (AMR), Advanced Audio Coding (AAC), Enhanced Variable Rate CODEC (EVRC), 3GPP, and the like. In one embodiment, audio260can be received, encoded, and communicated to multipoint server230in real-time or near real-time.

Conferencing device250can be hardware/software component associated with multipoint session permitting recording and presenting of audio. Device250can include, but is not limited to, audio input252, audio output254, interface256, digital signal processor (DSP), memory, and the like. Device250can include, but is not limited to, a VoIP telephone, telepresence system, and the like. In the instance, device250can be a hardware phone, software phone, and any combination thereof. In another instance, device250can be a videophone with conferencing capabilities. In yet another instance, device250can be a component of a videoconferencing system. It should be appreciated device250can include video components (e.g., camera, display) which can be selectively disabled.

Audio input252can be one or more hardware/software components able to receive sound (e.g., voice). Input252can include, but is not limited to, dynamic microphone, piezoelectric microphone, and the like. Audio input252can receive audio input from proximate participants which can be conveyed to server230as audio260. In one instance, audio260can be processed prior to transmission. Processing can include, but is not limited to, noise cancellation, encryption, and the like.

Audio output254can be one or more hardware/software components able to present sound. Audio output254can be an output device such as a loudspeaker. Audio output254can convey audio (not shown) received from multipoint server230associated with session236.

Interface256can be a hardware/software component for interacting with conferencing device250. Interface256can include, but is not limited to, input components, output components and the like. Input components can include, but is not limited to, keyboard, keypad (e.g., dialpad), and the like. Output components can include, but is not limited to, loudspeakers, display, and the like. In one embodiment, interface256can be utilized to manually join device250to a multipoint session236. In the embodiment, device250can be utilized for audio input and device210can be used for video input simultaneously.

Multipoint server230can be a hardware/software component for bridging video and audio associated with session236. Multipoint server230can include, but is not limited to, bridge manager232, presence engine234, A/V engine235, session236, configuration settings238, data store240, and the like. In one instance, multipoint server can be a component of a H.323 gatekeeper. In another instance, multipoint server can be a component of a multipoint control unit (MCU). In one instance, server230can be a network element within a local area network (LAN) and/or wide area network (WAN). In one embodiment, multipoint server230can be a component of IBM LOTUS UNIFIED COMMUNICATIONS AND COLLABORATION (UCC) software. It should be appreciated that components232-240can be optional components.

Bridge manager232can be a hardware/software component for managing session236control. Manager232functionality can include, but is not limited to, initiating session236, terminating session236, manipulating device210session interaction, handling device250session interaction, address translation, protocol conversion, call management, authentication, encryption, and the like. In one instance, manager232can automatically dial conferencing device250address and associate the device250with session236. In the instance, manager232can signal terminal device210to terminate audio input and trigger conferencing device250to initiate receiving audio input.

Presence engine234can be a hardware/software entity for tracking device210location. Presence engine234can monitor presence information237for each participant associated with device210. Engine234can be configured to respond to one or more participant states and/or locations. In one embodiment, engine234can trigger device210to terminate receiving of audio input from a participant if in proximity of device250and initiate collection of audio input if not in proximity of audio device250. In one instance, presence engine234can be an external resource (e.g., presence server).

Presence information237can be one or more datasets associated with terminal device210of session236. Presence information237can include, but is not limited to, location, user state, and the like. Location can include, but is not limited to, a geographic location, coordinates (e.g., Cartesian coordinates), a zone, and the like. Presence information237can be automatically populated by presence engine234. Information237can be used to determine if a terminal device is proximate to a conferencing device260. For instance, information237can be evaluated against known locations of conferencing devices to determine if a participant is close to a conference phone.

A/V engine235can be a hardware/software component for synchronizing audio260with video262. Engine235can utilize conventional and/or proprietary synchronization mechanisms, including, but not limited to, time-stamping, relative time-stamping, and the like. In one embodiment, engine235can create synchronized A/V246which can be conveyed to devices210,260. In the embodiment, synchronized A/V246can be conveyed in real-time to appropriate devices210,260. In one instance, engine235can utilize audio synchronizer functionality to enable audio-to-video synchronization (e.g., lip synchronization) to be achieved. Engine235can operate in real-time or near real-time permitting system200functionality to be transparent. Engine235can use one or more traditional and/or proprietary mechanisms for alleviating latency, jitter, and other synchronization problems associated with network connectivity.

In one embodiment, engine235can be utilized to determine an active speaker from an audio260stream. In the embodiment, engine235can employ one or more intelligent algorithms to determine a best choice active speaker for the audio260stream. Algorithms can perform audio analysis (e.g., speech processing) including, but not limited to, amplitude analysis, utterance duration, and the like.

Synchronized A/V246can be a dataset of synchronized audio and video generated by engine235. Synchronized A/V246can include, but is not limited to, an audio stream, a video stream, presence information, metadata, and the like. In one instance, A/V246can be individually communicated to appropriate devices210,260. In the instance, the video stream of A/V246can be conveyed to device210and the audio stream of A/V246can be communicated to conferencing device260simultaneously. That is, audio and video output is synchronized resulting in a solution similar to traditional multipoint sessions.

Configuration settings238can be one or more options for manipulating the behavior of server230. Configuration settings238can include, but is not limited to user configuration settings, system settings, and the like. Settings238can include, but is not limited to, session configuration, dial plans, call policies, Quality of Service (QoS) policies, alias information, and the like.

Data store240can be a tangible storage medium for persisting mapping242. Data store240can include, but is not limited to, Relational Database Management System (RDMBS), Object Oriented Database Management System (OODBMS), and the like. Data store240can be a component of a Network Attached Storage (NAS), Storage Attached Network (SAN), and the like.

Mapping242can be a dataset permitting linking of terminal device and conference device within session236. Mapping242can include, but is not limited to, terminal device identifier, terminal device location, terminal device properties, participant information, session information, conference device identifier, conference device location, conference device properties, and the like. Mapping242can be used to associate terminal device to a conference device to enable simultaneous use of terminal device (e.g., video) and conference device (e.g., audio). For example, mapping entry244can associate a tablet computer (e.g., Dev_A) with a conference phone (e.g., ConfD_A) to enable simultaneous use of tablet and phone resources. Mapping242can be automatically and/or manually established. In one instance, mapping242can be continually updated in real-time to enable rapid adaptation to changes in terminal device210location.

Terminal device210can be a computing device associated with a multipoint session. Terminal device210can be, but is not limited to, a desktop computer, a laptop computer, a tablet computing device, a mobile phone, a personal digital assistant (PDA), Voice over Internet Protocol (VoIP) capable device, and the like. Device210can include, but is not limited to, hardware211, software220, and interface224. Hardware205can include, but is not limited to processor212, non-volatile memory211, volatile memory213, bus214, video input215, video output216, GPS receiver217, transceiver218, and the like. Software220can include, but is not limited to, operating system221, multipoint application222, and the like. In one embodiment, terminal device210can be a laptop computer executing a multipoint application222. It should be appreciated that terminal device210can include audio input/output components, but have been selectively omitted for clarity.

Processor212can be a portion of a computing system able to execute instructions of a computing program (e.g., application222). Processor212can include, but is not limited to, central processing unit (CPU), microprocessor, graphics processing unit (GPU), and the like. Processor212can be utilized to process input from A/V input215and output from A/V output216. Processing can include, but is not limited to, noise cancellation, buffering, and the like.

Non-volatile memory211can be a static storage medium for persisting session data (e.g., configuration settings) associated with session236. Memory211can include, but is not limited to, flash memory, hard disk drive, and the like.

Volatile memory213can be a dynamic storage medium for temporarily persisting session data (e.g., video input) associated with session236. Memory213can include, but is not limited to, random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), and the like.

Bus214can be an interconnected subsystem for communicatively linking components212-218. Bus214can be serial and/or parallel permitting high speed communications to and from components212-218. In one instance, bus214can communicate data from components215,216to components212-214,218.

Video input215can be a hardware component for receiving video input from a participant. Video input215can be a component such as a charge-coupled device (CCD). In one embodiment, data from video input215can be received, encoded, and communicated in real-time or near real-time. In the embodiment, received input can be encoded via processor212and communicated using transceiver218in real-time to server230as video262.

Video output216can be a hardware component for presenting a video portion of synchronized A/V246associated with session236. Output216can receive A/V246from multipoint-server and present the video portion within output216. For example, video of a participant can be presented within an application222interface (e.g., interface224) upon output216. Output216can include, but is not limited to, computer display, display of a specialized communication device, and the like. Output216can conform to traditional and/or proprietary technologies including, but not limited to, Liquid Crystal Display (LCD), Organic Light Emitting Diode (LED), and the like.

GPS receiver217can be a device able to receive GPS data based on an automatically determined the location or a user specified location. GPS data can include, but is not limited to, mapping data, GPS coordinate data, and the like. In one instance, GPS receiver217can be an external device communicatively linked to device210.

Transceiver218can be a component permitting communication between components230,250. Transceiver218can be a computer networking transceiver, Radio Frequency (RF) transceiver, and the like.

Application222can be a software entity permitting communication of audio (not shown) and video262to multipoint server230. Application222can be responsive to device210location based on one or more customizable configurations. In one instance, device210can automatically terminate/activate an audio input associated with device210if proximity of device250changes. In one embodiment, termination/activation of audio can be based on utterances. In one configuration of the embodiment, application222can determine pauses in an audio stream received by an audio input component (e.g., microphone) and intelligently activate/terminate audio input. In the instance, an interface notification can be presented, alerting a participant that the audio stream of a proximate device250can be utilized. In one instance, application222can be IBM SAMETIME software.

Network270can be a hardware/software component for permitting communication between components210,230,250. Network270can include, but is not limited to, wired technologies, wireless technologies, and any combination thereof. Wired technologies can include, but is not limited to, twisted pair, coaxial cable, optical fiber, and the like. Wireless technologies can include, but is not limited to, cellular systems, Personal Communication Service (PCS) system, wireless local area network (WLAN) terrestrial microwave, and the like. Network270can utilize communication protocols including, but not limited to, Real-time Transport Protocol (RTP), RTP Control Protocol (RTCP), User Datagram Protocol, Internet Protocol, H.323, Session Initiated Protocol, and the like.

Drawings presented herein are for illustrative purposes only and should not be construed to limit the invention in any regard. System200can be a component of a distributed network, distributed computing environment, cloud computing environment, and the like. It should be appreciated that system200functionality can be implemented within an existing multipoint architecture. It should be noted that system200communication can be bi-directional and/or real-time. Further, system200can enable interaction with third party software and/or multipoint conferencing units which can lack video switching functionality.

In one instance, server230can automatically detect resource capacity limits and notify relevant participants through an interface notice presented upon device210(e.g., interface224). In the instance, the notice can present a participant with a suggested proximate conferencing device250. For example, if server230resources are limited, one or more participants can be notified that a conference phone associated with session236is proximate.

FIG. 3is a schematic diagram illustrating a set of interfaces310,340for improving multipoint session scalability for co-located participants in accordance with an embodiment of the inventive arrangements disclosed herein. Interface310,340can be present in the context of method100and system200. Interface310,340can be a screen associated with a configuration window. Interface310,340can be associated with client-side settings, server-side settings and the like. In one instance, interface310,340can be presented within interface224. In another instance, interface310,340can be presented within an interface (not shown) associated with server230. In one instance, interface310,340can be associated with one or more zones within a network configuration.

In interface310, conferencing device configuration can be established and/or manipulated. Manipulation can include, but is not limited to, adding a conferencing device, modifying a conference device configuration, deleting a conference device, and the like. In one instance, interface310can be a server-side setting associated with server230. In the instance, interface310can permit modification of mapping242.

In section312, available conference devices registered with system200can be presented. In the section312, conference device selection can be performed. Selection of a conference device can permit actions322-324to be initiated.

Element320can permit the automated and/or manual addition of a conferencing device. In one instance, element320can present a pop-up dialog enabling conferencing device details to be inputted. In another instance, element320can present a pop-up dialog presenting a series of automatically detected conferencing devices. If a conferencing device is added, the device can be registered with system200and presented within section312.

Element322can allow modification of registered conferencing devices through device selection in section312. Element222can enable dynamic conferencing device setting adjustments to be performed. In one instance, element222can present a pop-up dialog showing current conferencing device settings. In the instance, user input can be received to change conferencing device settings. For instance, if a conferencing device identity changes, element322can be utilized to manually update the setting to reflect the change.

Element324can permit de-registration of conferencing devices from system200. Element324can remove selected conferencing device in section312from being presented. In one instance, conferencing device settings can be removed if element324is selected. In another instance, conferencing device settings can be automatically persisted. For example, element324can be utilized to remove conferencing devices which have been removed and/or replaced due to device failure.

In interface340, terminal device configuration settings can be established and/or manipulated. In interface340, settings342-346can be utilized if a terminal device joins and/or leaves the proximity of a conference device. Settings342-346can be dynamically applied enabling flexible real-time configuration options. It should be appreciated that, interface340is not limited to terminal device settings and can include, session settings, conference device settings, and the like.

Setting342can enable the preservation of video mechanics if utilizing a proximate conferencing device. If setting342is enabled, system230can automatically detect the appropriate video implementation in use for session236. In one instance, if video switching, such as an “active speaker” functionality, is utilized within a session, video switching can be maintained utilizing setting342.

Setting344can permit user (e.g., participant) notification if audio inputs are changed due to proximity with a conferencing device. Setting344can include one or more options to notify a user if audio input is terminated and/or initiated on a terminal device. For instance, an audible sound (e.g., beep) can be presented if a user moves out of proximity of a conferencing device and audio input is initiated on the terminal device.

Setting346can present one or more proximate conferencing devices which can be available based on terminal device proximity. In one instance, setting346can permit manual selection of proximate conferencing devices. In another instance, setting346can allow automated selection of a conferencing device to be presented. For instance, setting346can present an alias (e.g., Conf. Room A) of proximate conferencing devices, enabling a participant to easily identify the conferencing device.

Drawings presented herein are for illustrative purposes only and should not be construed to limit the invention in any regard. Interface310,340can be a graphical user interface, text based user interface, voice user interface, mixed-mode interface, and the like. Functionality expressed within interface310,340can be present within file menus, context menus, menu bars, toolbars, and the like. Interface310,340can include one or more graphical user interface elements including, but not limited to, checkboxes, radio dialogs, combo boxes, and the like. Interface310,340can receive input and/or selection via traditional and/or proprietary mechanisms, including, keyboard, mouse, gesture, and the like.