Patent application EP20181116A (EP), filed 2020-06-19
Publication number: EP3754479A1
Title: SYSTEM AND METHOD FOR COMMUNICATING AMONG CLIENTS
Abstract: A communication system for exchanging video images with annotations is set out. The annotations may be time sensitive so as to fade after a preselected amount of time and not clutter up the screen on which the video images are being displayed. The annotations may be drawable entities made by gestures which, after being made, are read, decoded and stored for potential future reference.
IPC codes: G06F   3/    01            A I
G06F   3/  0488            A I
G06F  40/   169            A I
G11B  27/   031            A I
H04M   1/ 72439            A I
H04N  21/  4788            A I
H04N   7/    15            A I
Claims: number: 1
text: A computer-implemented process for communicating among a plurality of clients subscribing to common live feeds, each of the feeds comprising video channel and data track, and each of the clients comprising a video layer for displaying the video channel, a draw layer for displaying a drawable entity and a user interface layer for detecting an inputted gesture and storing it as a drawable entity, the process comprising:
number: 2
text: The process according to claim 1, wherein the drawable entity comprises at least one of a fading drawable entity, persistent drawable entity and point drawable entity.
number: 3
text: The process according to claims 1 and 2, wherein the fading pointer drawable entity and the point drawable entity comprise a time limit and the drawable entity automatically expires after an expiration of the time limit.
number: 4
text: The process according to claims 1-3, wherein the plurality of clients each comprise means for detecting gesture and storing and displaying the drawable entity.
number: 5
text: A computer-implemented process for communicating among a first and a second client over a feed, the feed including a video channel and data track, and each of the clients comprising a video layer for displaying the video channel, a draw layer for displaying a drawable entity and a user interface layer for detecting an inputted gesture and storing it as a drawable entity, the process comprising:
number: 6
text: The process according to claim 5, wherein the step of creating further comprises the steps of determining the type and screen coordinates of the gesture.
number: 7
text: The process according to claims 5-6, wherein the screen coordinates are normalized screen coordinates.
number: 8
text: The process according to claims 5-7, wherein the step of storing the drawable entity further comprises the step of storing the entity type, entity lifetime and screen coordinates of the drawable entity.
number: 9
text: The process according to claims 5-8, wherein the gesture type is single touch, a pan or a persistent line.
number: 10
text: A system for communicating among a plurality of clients subscribing to common live feeds, each of the feeds comprising video channel and data track, and each of the clients comprising a video layer for displaying the video channel, a draw layer for displaying a drawable entity and a user interface layer for detecting an inputted gesture and storing it as a drawable entity, the process comprising:
number: 11
text: The system according to claim 10, wherein the drawable entity comprises at least one of a fading drawable entity, persistent drawable entity and point drawable entity.
number: 12
text: The system according to claims 10-11, wherein the fading pointer drawable entity and the point drawable entity comprise a time limit and the drawable entity automatically expires after an expiration of the time limit.
number: 13
text: The system according to claims 10-12, wherein the plurality of clients each comprise means for detecting gesture and storing and displaying the drawable entity.
number: 14
text: A system for communicating among a first and a second client over a feed, the feed including a video channel and data track, and each of the clients comprising a video layer for displaying the video channel, a draw layer for displaying a drawable entity and a user interface layer for detecting an inputted gesture and storing it as a drawable entity, the process comprising:
Office action: search_report, dated 2020-12-23
Rejection reasons: inventive_step_epc_56
Prior art cited: ref_id: US9307293B2
ref_type: patent
severity: background
ref_id: US9654727B2
ref_type: patent
severity: background
ref_id: US9113033B2
ref_type: patent
severity: background
ref_id: US9686497B1
ref_type: patent
severity: background
ref_id: US2018349014A1
ref_type: patent
severity: background
ref_id: US2015220504A1
ref_type: patent
severity: background
ref_id: - "Open Source Based Online Map Sharing to Support Real-Time Collaboration", OSGEO JOURNAL, vol. 10
ref_type: npl
severity: background
ref_id: - "A WebRTC-Based Video Conferencing System with Screen Sharing published", 2ND IEEE INTERNATIONAL CONFERENCE ON COMPUTER AND COMMUNICATIONS (ICCC, 2016
ref_type: npl
severity: background
ref_id: - ZHENYAO MO ET AL: "SmartCanvas", 2005 INTERNATIONAL CONFERENCE ON INTELLIGENT USER INTERFACES. IUI 05. SAN DIEGO, CA, JAN. 9 - 12, 2005; [ANNUAL INTERNATIONAL CONFERENCE ON INTELLIGENT USER INTERFACES], NEW YORK, NY : ACM, US, 10 January 2005 (2005-01-10), pages 239 - 243, XP058319625, ISBN: 978-1-58113-894-8, DOI: 10.1145/1040830.1040881
ref_type: npl
severity: background
ref_id: US2018348955A1
ref_type: patent
severity: background
ref_id: US2019182454A1
ref_type: patent
severity: background
ref_id: US2011164163A1
ref_type: patent
severity: background
ref_id: US2015194187A1
ref_type: patent
severity: background
ref_id: US2011107238A1
ref_type: patent
severity: background
ref_id: - ZHANG JIN-KAI ET AL: "Collaborative Interaction for Videos on Mobile Devices Based on Sketch Gestures", JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY, SCIENCE PRESS, BEIJING, CN, vol. 28, no. 5, 17 September 2013 (2013-09-17), pages 810 - 817, XP035351307, ISSN: 1000-9000, [retrieved on 20130917], DOI: 10.1007/S11390-013-1379-4
ref_type: npl
severity: background
ref_id: - SEUNGWON KIM ET AL: "Comparing pointing and drawing for remote collaboration", 2013 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR), 1 October 2013 (2013-10-01), pages 1 - 6, XP055733060, DOI: 10.1109/ISMAR.2013.6671833
ref_type: npl
severity: background
ref_id: - FUSSELL S R ET AL: "Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks", HUMAN-COMPUTER INTERACTION, LAWRENCE ERLBAUM ASSOCIATES, HILLSDALE, NJ, US, vol. 19, 1 September 2004 (2004-09-01), pages 273 - 309, XP008132713, ISSN: 0737-0024
ref_type: npl
severity: background
ref_id: - CHANG YUN SUK ET AL: "Gesture-based augmented reality annotation", 2017 IEEE VIRTUAL REALITY (VR), IEEE, 18 March 2017 (2017-03-18), pages 469 - 470, XP033084030, DOI: 10.1109/VR.2017.7892383
ref_type: npl
severity: background
ref_id: - OMID FAKOURFAR ET AL: "Stabilized Annotations for Mobile Remote Assistance", HUMAN FACTORS IN COMPUTING SYSTEMS, ACM, 2 PENN PLAZA, SUITE 701 NEW YORK NY 10121-0701 USA, 7 May 2016 (2016-05-07), pages 1548 - 1560, XP058257259, ISBN: 978-1-4503-3362-7, DOI: 10.1145/2858036.2858171
ref_type: npl
severity: background
ref_id: - JIAZHI OU ET AL: "Gestural communication over video stream", ICMI'03. 5TH. INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES. VANCOUVER, BRITISH COLUMBIA, CANADA, NOV. 5 - 7, 2003; [INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES], NEW YORK, NY : ACM, US, 5 November 2003 (2003-11-05), pages 242 - 249, XP058225613, ISBN: 978-1-58113-621-0, DOI: 10.1145/958432.958477
ref_type: npl
severity: background
ref_id: US2011107238
ref_type: patent
severity: background
ref_id: US2011164163
ref_type: patent
severity: background
ref_id: US2015194187
ref_type: patent
severity: background
ref_id: US2018348955
ref_type: patent
severity: background
ref_id: US2019182454
ref_type: patent
severity: background
ref_id: - ZHANG JIN-KAI ET AL, "Collaborative Interaction for Videos on Mobile Devices Based on Sketch Gestures", JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY, SCIENCE PRESS, BEIJING, CN, vol. 28, no. 5, doi:10.1007/S11390-013-1379-4, ISSN 1000-9000, (20130917), pages 810 - 817, (20130917), XP035351307 [I] 1-5,10-14 * the whole document * [Y] 6-9
ref_type: npl
severity: obviousness
ref_id: - SEUNGWON KIM ET AL, "Comparing pointing and drawing for remote collaboration", 2013 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR), (20131001), doi:10.1109/ISMAR.2013.6671833, pages 1 - 6, XP055733060 [I] 1-5,10-14 * the whole document * [A] 6-9
ref_type: npl
severity: background
ref_id: - FUSSELL S R ET AL, "Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks", HUMAN-COMPUTER INTERACTION, LAWRENCE ERLBAUM ASSOCIATES, HILLSDALE, NJ, US, (20040901), vol. 19, ISSN 0737-0024, pages 273 - 309, XP008132713 [I] 1-5,10-14 * the whole document *
ref_type: npl
severity: background
ref_id: - CHANG YUN SUK ET AL, "Gesture-based augmented reality annotation", 2017 IEEE VIRTUAL REALITY (VR), IEEE, (20170318), doi:10.1109/VR.2017.7892383, pages 469 - 470, XP033084030 [Y] 6-9 * the whole document * [A] 1-4,10-14
ref_type: npl
severity: obviousness
ref_id: - OMID FAKOURFAR ET AL, "Stabilized Annotations for Mobile Remote Assistance", HUMAN FACTORS IN COMPUTING SYSTEMS, ACM, 2 PENN PLAZA, SUITE 701 NEW YORK NY 10121-0701 USA, (20160507), doi:10.1145/2858036.2858171, ISBN 978-1-4503-3362-7, pages 1548 - 1560, XP058257259 [I] 1-5,10-14 * the whole document *
ref_type: npl
severity: background
ref_id: - JIAZHI OU ET AL, "Gestural communication over video stream", ICMI'03. 5TH. INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES. VANCOUVER, BRITISH COLUMBIA, CANADA, NOV. 5 - 7, 2003; [INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES], NEW YORK, NY : ACM, US, (20031105), doi:10.1145/958432.958477, ISBN 978-1-58113-621-0, pages 242 - 249, XP058225613 [A] 1-14 * the whole document *
ref_type: npl
severity: background
Final disposition: pending (2020-12-23)