Exhibit 11

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 1 of 36 PageID #:

4063
| HAO WAKATI AKITOA US009800708B2 UTAN U MAMA AT 

( 12 ) United States Patent 

Lim et al . ( 10 ) Patent No . : US 9 , 800 , 708 B2 

( 45 ) Date of Patent : * Oct . 24 , 2017 

( 54 ) OPERATION METHOD AND SYSTEM OF 

MOBILE TERMINAL 

( 71 ) Applicant : Samsung Electronics Co . , Ltd . , Gyeonggi - do ( KR ) 

( 52 ) U . S . CI . CPC . H04M 1 / 72527 ( 2013 . 01 ) ; H04M 1 / 274516 

( 2013 . 01 ) ; H046 88 / 02 ( 2013 . 01 ) ; H04M 

1 / 7253 ( 2013 . 01 ) ; H04M 1 / 72552 ( 2013 . 01 ) 

( 58 ) Field of Classification Search 

CPC . . . . . . . HO4M 1 / 72527 ; H04M 1 / 274516 ; H04M 

1 / 72552 ; H04M 1 / 7253 ; H04W 88 / 02 

( Continued ) 

( 72 ) Inventors : Hun Lim , Gyeonggi - do ( KR ) ; 

Gene - Moo Lee , Gyeonggi - do ( KR ) ; 

Jin - Wook Lee , Gyeonggi - do ( KR ) ; 

Je - Hyok Ryu , Gyeonggi - do ( KR ) ; 

Woo - Jin Park , Gyeonggi - do ( KR ) ; 

Jin - Hyoung Kim , Gyeonggi - do ( KR ) ; 

Shin - Il Kang , Gyeonggi - do ( KR ) 

( 73 ) Assignee : Samsung Electronics Co . , Ltd ( KR ) 

( 56 56) )References Cited 

U . S . PATENT DOCUMENTS 

5 , 920 , 826 A * 7 / 1999 Metso . . . . . . . . . . . . H04M 1 / 274525 455 / 466 6 , 968 , 209 B1 * 11 / 2005 Ahlgren . . . . . . . . . . . . G06F 21 / 64 

370 / 350 

( Continued ) 

( * ) Notice : Subject to any disclaimer , the term of this 

patent is extended or adjusted under 35 

U . S . C . 154 ( b ) by 0 days . 

This patent is subject to a terminal dis 

claimer . FOREIGN PATENT DOCUMENTS 

CN ( 21 ) Appl . No . : 15 / 132 , 877 CN 

1525723 9 / 2004 

1971520 5 / 2007 

( Continued ) ( 22 ) Filed : Apr . 19 , 2016 

( 65 ) Primary Examiner – Khai M Nguyen ( 74 ) Attorney , Agent , or Firm — The Farrell Law Firm , P . C . 

Prior Publication Data 

US 2016 / 0234370 A1 Aug . 11 , 2016 

Related U . S . Application Data 

( 63 ) Continuation of application No . 14 / 252 , 310 , filed on 

Apr . 14 , 2014 , now Pat . No . 9 , 332 , 110 , which is a 

( Continued ) 

( 30 ) Foreign Application Priority Data 

( 57 ) ABSTRACT 

An operation method and mobile terminal connected to an 

electronic device . The operation method includes extracting 

a piece of content displayed on a portion of a display of the 

electronic device , corresponding to a user selection , from 

among multiple pieces of content displayed on the display ; 

displaying a function of the mobile terminal corresponding 

to the extracted piece of content ; and delivering , to the 

mobile terminal , the extracted piece of content and a com 

mand of the function of the mobile terminal that will execute 

the extracted piece of content . 

Feb . 13 , 2009 

Mar . 31 , 2009 

( KR ) . . . . . . . . . . . . . . . . . 10 - 2009 - 0012155 

( KR ) . . . . . . . . . . . . . . . . . . . . . . . . 10 - 2009 - 0027864 

( 51 ) Int . Cl . H04B 1 / 38 

H04M 1 / 725 ( 2015 . 01 ) ( 2006 . 01 ) 

( Continued ) 16 Claims , 23 Drawing Sheets 

INTERACTIVE 

SERVICE MODE 

11 

NO CONTENT SELECTED ? 

YES 

EXTRACT SELECTED CONTENT 

AND IDENTIFY TYPE 

PROVIDE RELEVANT 

PHONE FUNCTION 15 

17 

NO APPLICATION 

SELECTED ? 

YES 

CONFIGURE AND DELIVER EXECUTION 

DATA CORRESPONDING SELECTED 

PHONE FUNCTION AND COMMAND 

AND INCLUDING EXTRACTED CONTENT 

- 19 

END 

HW103-00082269

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 2 of 36 PageID #:

4064
US 9 , 800 , 708 B2 

Page 2 

Related U . S . Application Data 

continuation of application No . 12 / 706 , 591 , filed on 

Feb . 16 , 2010 , now Pat . No . 8 , 725 , 209 . 

( 51 ) Int . Ci . H04W 88 / 02 ( 2009 . 01 ) 

H04M 1 / 2745 ( 2006 . 01 ) 

( 58 ) Field of Classification Search 

USPC . . . . . . . . . . . . . 455 / 557 , 550 . 1 , 566 , 414 . 1 , 435 . 1 

See application file for complete search history . 

2008 / 0122796 A15 / 2008 Jobs et al . 2008 / 0123624 A15 / 2008 Strandell et al . 2008 / 0160956 A1 7 / 2008 Jackson et al . 2008 / 0181197 A1 * 7 / 2008 Yin . . . . . . . . . . . . H04L 12 / 66 

370 / 352 

2008 / 0195997 A1 * 8 / 2008 Herberger . . . . . . . . . . . . . G06F 8 / 34 

717 / 100 

2008 / 0250100 A1 * 10 / 2008 Hatanaka . . . . . . . . . G06F 17 / 30799 

709 / 203 

2008 / 0263020 A1 * 10 / 2008 Takehara . . . . . . . . . . G06F 17 / 30864 

2009 / 0055510 A 2 / 2009 Svendsen 

2009 / 0104938 AL 4 / 2009 Tang 

2010 / 0223314 A1 9 / 2010 Gadel et al . 2011 / 0098025 A14 / 2011 Ho et al . 2012 / 0297425 A1 * 11 / 2012 Sathiananthan . . . HO4N 21 / 2343 

725 / 63 

( 56 ) References Cited 

U . S . PATENT DOCUMENTS 

FOREIGN PATENT DOCUMENTS 8 , 725 , 209 B2 * 5 / 2014 Lim . . . . . . . . . . . . . . . . H04M 1 / 274516 

455 / 414 . 1 

9 , 332 , 110 B2 * 5 / 2016 Lim HO4M 1 / 274516 

2004 / 0092266 A1 * 5 / 2004 Olrik . . . . . . . . . . . . . . . . . H04M 1 / 72525 

455 / 445 

2005 / 0070224 A1 * 3 / 2005 Ho . . . . . . . . . . . . . . . . HOLM 1 / 72527 

455 / 41 . 2 

2008 / 0003996 A11 / 2008 Strandell et al . 

CN 

KR 

KR 

101079828 

1020050098525 

1020070042666 

WO 2005 / 088947 

WO 2007 / 102045 

11 / 2007 

4 / 2007 

4 / 2007 

9 / 2005 

9 / 2007 

WO 

WO 

* cited by examiner 

HW103-00082270

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 3 of 36 PageID #:

4065
atent Oct . 24 , 2017 Sheet 1 of 23 US 9 , 800 , 708 B2 

100 200 

COMPUTER MOBILE TERMINAL 

INTERACTIVE SERVICE 

APPLICATION - 1401 PHONE APPLICATION 220 

CONTENT EXTRACTOR PHONE AGENT | 210 

PHONE SERVICE 

APPLICATION 

DEVICE AGENT 110 

FIG . 1 

U.S.

k

120

HW103-00082271

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 4 of 36 PageID #:

4066
- 

- 

- 

- 

L 

- 

- 

- 

- 

- 

- 

- - 

INTERACTIVE SERVICE APPLICATION 

COMPUTER 

- - 

100 

- 140

atent 

- - 

TOOL BAR-1

CONTEXT MENU 

- 

142 

- 

- 

- - 

- - 

1 

- 

- 

- 

CONTENT EXTRACTOR 

- 

- 130

- 

200 

- 

120 - 

- 

- 

- 

- 

- 

- 

+ 

- 

- 

121 

122 

123 

MOBILE TERMINAL 

125 

124 

- - 

Oct . 24 , 2017 

- 

- - 

- 

- - - - 

- 

- - 

- 

- 

- 

- 

CALL SENDER /RECEIVER 

- 

SMS SENDER RECEIVER 

MMS SENDER /RECEIVER 

IMAGE CAPTURE 

MEMO VIEWER 

- 

PHONE APPLICATION 223 

- 

- 

- - 

- 

- 

- 

- 

221 

222 

224 

225 

- 

- 

- - 

- 

126 

127 

- 

- 

- 

- - 

- 

- 

- - 

- 

- - 

CALL SENDER :RECEIVER 

FTP SERVER 

MEMO WRITER 

- 

SMS SENDERI RECEIVER 

FTP CLIENT 

MMS SENDER /RECEIVER 

MEMO VIEWER 

- 

- 

PHONE SERVICE APPLICATION 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Sheet 2 of 23 

- 

- 

- - 

- 

- 

- - 

- 

- 

- - 

- 

- 

- - 

- 

- 

- - 

- 

- 

- 

DEVICE AGENT

112 

PHONE AGET

211 

212 

- 

- 

- 

111 

- 

- 

- 

- - 

1 

- 

- 

- 

- 

- 

- 

- 

- 

210 

- 

- 

110 

- 

- 

CONNECTION MANAGER 

- 

CONNECTION MANAGER 

- 

PHONE BOOKMANAGER 

- 

MANAGER 

- 

- 

- 

- 

- 

- L 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

+ 

+ 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

FIG .2

U.S. US 9 , 800 , 708 B2 

|

|

| | FOOS 3NOHa] L

»

|

| | | | | |

an

| 1001

' NOI VOMddY

HW103-00082272

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 5 of 36 PageID #:

4067
U . S . Patent Oct . 24 , 2017 Sheet 3 of 23 US 9 , 800 , 708 B2 

INTERACTIVE 

SERVICE MODE 

< CONTENT SELECTED ? > 

LYES 

EXTRACT SELECTED CONTENT 

AND IDENTIFY TYPE F - 13 

PROVIDE RELEVANT 

PHONE FUNCTION 

17 NO V 

APPLICATION 

SELECTED ? 

YES 

CONFIGURE AND DELIVER EXECUTION 

DATA CORRESPONDING SELECTED 1 - 19 

PHONE FUNCTION AND COMMAND 

AND INCLUDING EXTRACTED CONTENT 

END 

FIG . 3 

19

d

HW103-00082273

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 6 of 36 PageID #:

4068
U . S . Patent Oct . 24 , 2017 Sheet 4 of 23 US 9 , 800 , 708 B2 

INTERACTIVE 

SERVICE MODE 

EXECUTION DATA 

RECEIVED ? 

YES 

IDENTIFY PHONE FUNCTION 

AND COMMAND CORRESPONDING F33 

TO EXECUTION DATA 

EXECUTE PHONE FUNCTION AND 

COMMAND USING CONTENTINCLUDED - 35 

IN EXECUTION DATA 

END 

FIG . 4 

3]

( }

HW103-00082274

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 7 of 36 PageID #:

4069
U . S . Patent Oct . 24 , 2017 Sheet 5 of 23 US 9 , 800 , 708 B2 

200 100 

MOBILE 

TERMINAL COMPUTER 

PHONE EVENT 

DELVER PHONE EVENT ( 73 ) 

DISPLAY PHONE EVENT 75 

RESPONSE 

EXECUTE PHONE SERVICE APPLICATION 

CORRESPONDING TO PHONE EVENT ( 79 ) 

FIG . 5 

71

L_77

me

HW103-00082275

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 8 of 36 PageID #:

4070
230 

240 

? 130 

atent 

120 

110 

210 

220 

USER 

CONTENT 

COMPUTER 

SERVICE 

DEVICE 

PHONE 

PHONE 

SCREEN |EXTRACTOR APPLICATION 

AGENT 

AGENT 

APPLICATION 

PHONE BOOKREQUEST (301)

SELECT WEBCONTENT AND BAR TOOLINVOKE AND

PROVIDE PHONEBOOK ANDCALL HISTORY (303)

EXTRACT AND

CONTEXT MENU(305 )

INFORMATION CONVERT TEXT .CONTENT (WEB NUMBER PHONE NAM ,13077 (FIC HYPERLINK .

SIMPLETRING ,UL.

DELIVER DATAOFECH 

IMAGE ,ETC.)( 309

APPLICATION (SMS,M. CALL ,MEMO)( 311PHONE NUMBERLIST (315)ADDITIONAL INPUT,RECIPIENT SELECTAPPLICATION DATA ,ANDRECIPIENT 

APPLICATION FUNCTIONAL )319 (INFORMATION 

" DATAAND RECIPIENT 

INFORMATION (321 )JINVOKE FUNCTION AND

DELIVER DATA(323 ).

Oct . 24 , 2017 Sheet 6 of 23 

EXECUTE SMS,M.MEMO ANDCALL (325)

DISPLAY RESULTON VI(329) 

ACK (327)

FIG .6

U.S. US 9 , 800 , 708 B2 HOV (S26) ONY OWS ‘SIN SLNO3X3¥3ANI0 IN NO ONY NOILLONAS LN3IdIOIY NOILYONdd¥ NO NOLLYWHONI T¥O “SWIN ILVONdd¥ NOILWONdd¥ HOV3 Y3AI130 (608) ‘JOVI “WA JdWIS HAGWNN 3NOHd (O13 “WNITY3dAH NOILWAUOsH INAINOO 83M Tigg AYOLSIH ONY NOOB 3NOItd JaIAOud ONY CNY ev (IP8) YOOR SNOHd NOILVONdd¥ NEES

HW103-00082276

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 9 of 36 PageID #:

4071
U . S . Patent Oct . 24 , 2017 Sheet 7 of 23 US 9 , 800 , 708 B2 

COMPUTER ( 100 ) MOBILE TERMINAL ( 200 ) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

- - - - 

- - - 

- DEVICE AGENT ( 110 ) - - - 

112 

PHONE AGENT ( 210 ) 

212 - - - - 

- 111 - - 211 - 

- - - - 

- - - 

- PHONE BOOK - 

MANAGER 

CONNECTION 

MANAGER 

CONNECTION 

MANAGER 

PHONE BOOK ) 

MANAGER 

- 

- 

- Hello ( 401 ) - 

- 

Hello ACK ( 403 ) 

PHONE BOOK REQUEST ( 405 ) - 

- 

- 

- 

- DELIVER PHONE BOOK ( 407 ) - 

- 

- 

- LAST CALL HISTORY ( 409 ) 

FREQUENT CALL HISTORY ( 411 ) - 

- 

- 

- - 

- - 

- 1 ACK - 

- + 

- 

- 

- 

- 

- 

- - 

- - 

- - 

- - - - - - - - - - - - - - L - — — - - - - - - - - 

FIG . 7 

)

|

|

| |

|

| |

! !

|

T T

I |

| |

|

_

A

4

| |

4

HW103-00082277

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 10 of 36 PageID #:

4072
atent Oct . 24 , 2017 Sheet 8 of 23 US 9 , 800 , 708 B2 

COMPUTER ( 100 ) MOBILE TERMINAL ( 200 ) - - - - - - - - - - - - - - - - - - - - - 

- 

- DEVICE AGENT ( 110 ) - 

111 112 

PHONE AGENT ( 210 ) 

211 212 - 

- - - 

- - - 

- - - PHONE BOOK 

MANAGER 

CONNECTION - - 

MANAGER 

CONNECTION 

MANAGER 

PHONE BOOK 

| MANAGER - 

- - 

- 

- - 

- - - 

- 

- - 

- - 

- - 

Stop ( 451 ) - 

- 

- Stop ACK ( 453 ) - 

- 

- 

- 

- 

- 

j - - - - - - - - 

FIG . 8 

U.S.

71 rae

|

|

|

|

{

i

A

III||1|I{11|I

{

|!

I

i{{II

HW103-00082278

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 11 of 36 PageID #:

4073
U . S . Patent Oct . 24 , 2017 Sheet 9 of 23 US 9 , 800 , 708 B2 

COMPUTER ( 100 ) MOBILE TERMINAL ( 200 ) - - - - - - - - - - - - - - - - - - - - 

- 

- 120 110 - 210 220 - 

- 

- PHONE SERVICE - 

APPLICATION 

DEVICE 

AGENT 

PHONE 

AGENT 

PHONE - 

APPLICATION - 

- - 

- - 

- 

- EXECUTION DATA 

( CONTENT , COMMAND ) ( 501 ) - 

- 

- 

- 

- 

- 

- 

- 

DELIVER EXECUTION DATA ( 503 ) 

SEND COMMAND ( 505 ) - 

- 

- 

- 

- 

- - 

- 

- 

- 

- PROCESS 

CONTENT ( 507 ) - 

- 

- 

- ACK ( 509 ) - 

- 

- - 

- 

- 

- 

- 

- 

- - - - - - - - - - - 

FIG . 9 

7 7

| } {

| | | |

i | |

{

1 I

al I |

|

{ |

1

!

1

i

| {

| Po:

HW103-00082279

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 12 of 36 PageID #:

4074
U . S . Patent 

100 )(COMPUTER 

200 )(TERMINAL MOBILE 

- 

- 

- 

- 

- 

- 

- 

- 

L 

- 

- 

- 

- 

- 

- 

- 

I 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

110 )AGENT (DEVICE 

111 

112 

- 

- 

- 

130 

- 

121 

- - 

- 

- 

210 )AGENT (PHONE 

211 

212 

222 

SENDERII CALLBOOKI |PHONE CONNECTION (RECEIVER MANAGER 

- 

- 

- - 

CONTENT EXTRACTOR 

- 

BOOK PHONE /|SENDER CALL MANAGER RECEIVER 

CONNECTION MANAGER 

- 

Oct . 24 , 2017 

- - 

551 )NUMBER (DELIVER 

- 

— 

- - 

REQUEST (55CALL 

- 

- 

555 )REQUEST (CALL 

- 

- - 

- 

- 

- 

- 

557 )REQUEST (CALL 

- - 

- 

- - 

- 

- 

- 

- 

- 

- 

) 559 (ACK CALL 

- 

Sheet 10 of 23 

- 

- 

+ - 

- 

1 

- 

— 

- 

. 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

10 .FIG

US 9 , 800 , 708 B2 

DEVICE (110)

CALL PHONE BOOK

MANAGER

DEUIVER (951)

CALL (553)

1\

|l

|

|

MOBILE (200)

PHONE (210) _—|

{5

CONNECTION] | |CALL SENDER/ MANAGER RECEIVER

I

CALL (559)

CALL (557)

CALL! (655)

||

|

|

eK

HW103-00082280

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 13 of 36 PageID #:

4075
U . S . Patent 

COMPUTER (100)

MOBILE TERMINAL (200)

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

L 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

- 

- 

DEVICE AGENT(110 )

111 

112 

AGENT 210)PHONE (

211 

212 

- 

- 

130 

121 

222 

- 

- 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

CONTENT EXTRACTOR 

SENDER CALL/RECEIVER 

CONNECTION IPHONE BOKMANAGER MANAGER 

IPHONE BOK)CONNECTION MANAGER |

SENDER CALL/TI RECEIVER 

- - 

- - 

- 

- 

- - 

- 

- - 

- 

- 

- 

- 

- - 

Oct . 24 , 2017 

- 

- - 

- 

- 

- 

- 

- 

- 

1 

- 

DELIVER (601)NAME 

SEARCH FORNUMBER (603)NUMBER LIST(605 )

REQUEST 609)CALL (

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

CALL REQUEST (611)

- - 

- 

- 

- 

- 

REQUEST (613)CALL 

Sheet 11 of 23 

- 

- 

- 

- 

- 

- 

- 

- 

- 

615 )CALLACK (

- 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

* 

L 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

FIG 11.

US 9 , 800 , 708 B2 

TWO

(149)

(609)

(609) Y3aWnn

YO4 (109) YSANS0

HSOWNYW | SNOHd TWO YIOVNVN TWO] |POO8

INOHd (O11) JOIAZO Le JIGOW

HW103-00082281

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 14 of 36 PageID #:

4076
COMPUTER (100)

U . S . Patent 

MOBILE TERMINAL (200)

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

- 

DEVICE AGENT(110 )

111 

112 

PHONE AGET(210 )

211 

212 

- - 

- 

- 

130 

223 

- 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

CONTENT EXTRACTOR 

- 

SMS RECEIVER 

IPHONE BOKCONNECTION MANAGER 

CONNECTION PHONEBOKI MANAGER |

SMSENDER /RECEIVER 

- 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

Oct . 24 , 2017 

- - 

- 

- 

- 

- 

- 

DELIVER 

TEXT (651)JSEARCH FOR

NUMBER (653)455 )NUMBER (DELIVER 

SMSEND REQUEST (657)

- 

- 

- 1 

/ 

- 

— 

1 SMSEND REQUEST (659)J

- - 

SMSEND REQUEST (661)

Sheet 12 of 23 

- 

- 

- 

- 

- 

- 

- 

SMS ACK(663 )

- 

- - - - - 

- 

- 

- 

1 

- 

LE 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

FIG .12

US 9 , 800 , 708 B2 

!

HH

id

{

(199) ANAS SWS

!

|e

GNIS SWS

(SSB) U3AI1HG

! YOI HOUVES (1S9)t|

HA9VNYW ysovnyw

| INOLLO3SNNOD| | | ENOILOINNOD} JNOHd

ool

(012) SNOHd (O11) JOIAZC

| Se (002)

HW103-00082282

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 15 of 36 PageID #:

4077
( 100)COMPUTER 

200 )TERMINAL (MOBILE - 

U . S . Patent 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

G 

- 

- 

- - 

- 

- 

AGENT 110)DEVICE (

- - 

- 

- 

210 )AGENT( PHOE 

211 

- 

- 

- 

130 

123 

111 

126 

112 32 

212 

225 

- 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

CONTENT EXTRACTOR 

CONNECTION BOOKPHONE (MANAGER 

- 

IMAGE 1CAPTURE 

???? WRITER 

BOOK PHONECONNECTION | MANAGER 

MEMO VIEWER 

- 

- 

- 

- 

- 

- 

- 

- 

- - - - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Oct . 24 , 2017 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

DELIVER URL

, TIMEJDELIVER URLTIME 901)(

ADDITIONAL IMAGE(903 )AND

INPUT 905)USER (INFORMATION MEMODELIVER IMAGE 997)(INPUT USER,TIME (URL.

MEMO REQUEST (909)J( 911)MEMO ACK( 913)INFORMATION MEMODE?IVER 

COMMAND (915)MEMO VIEW

ACK (917)DELIVERY INFORMATION MEMO

- - 

- 

- 

- 

- 

- 

Sheet 13 of 23 

- 

- 

- 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

13 FIG.

US 9 , 800 , 708 B2 

MOBILE

(200)

PHONE

(210)

CONNECTION]

BOOK MANAGER

peewee

URL/

TIME

(901)

q

USER

=

RL.

TIME.

USER

IMAGE)

(9¢

7)

MEMO MEMO

(911)

DEL

I

IVER

MEMO

($

MEMO

1

3)

MEMO ¢

VIEW

(915)

|

|

INFORMATION

ACI

(917)

HW103-00082283

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 16 of 36 PageID #:

4078
U . S . Patent Oct . 24 , 2017 Sheet 14 of 23 US 9 , 800 , 708 B2 

MOBILE TERMINAL ( 200 ) COMPUTER ( 100 ) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

- - 

220 210 110 120 - - 

- - 

- - 

- - 

- PHONE 

APPLICATION 

PHONE 

AGENT - DEVICE 

AGENT 

PHONE SERVICE 

APPLICATION - - 

- - 

- - 

- - 

- - 

- - 

- 

DELIVER PHONE INFORMATION - ( CONTENT , COMMAND ) ( 951 ) 

DELIVER PHONE INFORMATION ( 963 ) 

- - 

- - 

- 

- 

- + 

- - 

- - 

- - SEND COMMAND ( 955 ) - - 

- - 

- - PROCESS 

CONTENT ( 957 ) - - 

- 

- 

ACK ( 959 ) - - 

- 

- - 

- 

- 

- - 

- - 

- - 

- 

- - — - - - - - - - - - - - - - - - - - - - - - 

FIG . 14 

7

i

|

t

I

|

A

HW103-00082284

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 17 of 36 PageID #:

4079
U . S . Patent 

100 )COMPUTER (

200 )MOBILE TERMINAL (

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

AGENT (110)DEVICE 

111 

) 12(AGENI DEVICE 

AGENT (210)PHONE 

AGEN PHONE 

- 

127 

130 

125 

- 

127 

112 

211 211 

212 

221 

- 

i 

- 

FTP 

CONTENT EXTRACTOR 

a man

- 

MEMO VIEWER 

FTD CLIENT 

BOOK CONNECTION PHONEMANAGER |

BOOK PHONECONNECTION MANAGER |

- - 

SERVER 

- 

Oct . 24 , 2017 

- 

- 

- 

- 

- 

- 

VIEW COMMAND (70

- - 

- 

- 

- 

- 

- 

MEMO FILE703 )REQUEST (

- - 

- 

- - 

- 

REQUEST (705)NEMO FILEMEMO FILE(707 )DELIVER 

- - 

ACQUIRE FILEACK (709)

- - - 

Sheet 15 of 23 

- 

- 

- 1 

- - 

- - - 

MEMO (711)DISPLAY 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

L 

- 

- 

15 FIG.

US 9 , 800 , 708 B2 

(202) OWS Y3AITSO

|

(S02) OWSH

HADVNVAY

YOO NOILOSINNOD

(012)

JTIGON

(112) A¥1dSI0

HIOYNVN

NOILOANNOO] INOHd

q

(OL!) JOIAIA

HW103-00082285

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 18 of 36 PageID #:

4080
U . S . Patent 

100 )COMPUTER (

200 )MOBILE TERMINAL (

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

AGENT (110)DEVICE 

111 

112 

| 

AGENT (210)PHONE 

211 

212 

- 

130 

- 

125 

127 

221 

- - 

E 

B 

- 

CONTENT EXTRACTOR 

MEMO VIEWER 

- 

FTD CLIENT 

CONNECTION BOOKPHONE MANAGER |

BOOK PHONECONNECTION MANAGER |

FTP SERVER 

- - 

- 

- 

- 

Oct . 24 , 2017 

- 

- 

- 

- - 

VIEW 

COMMAND (751 )ACTIVATE FTP

CLIENT 753)(

- - - - 

FILE ACQUIRE HEQUEST (755)

- 

MEMO FILEREQUEST (757)

- - - 

REQUEST (759).ACQUIRE FILE

- 1 

- - 

MEMO FILERESPONSE (761)

- 

DELIVER MEMO 763)(

Sheet 16 of 23 

- 

- 

- - 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

DISPLAY MEMO(765 )

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

FIG .16

US 9 , 800 , 708 B2 9T

PO 71

(S9/) AV1dSIA

||

| || t |

>

OWS

« SUINDOV JS rot

OWS (GSZ) 315

bo LN3MNO1 dlj SLVAILOV

|

|[Tusovnww Tasovnvw

| [NOILOINNOO]

(012) ANOHd (O11)

Lo~U J Loo

(002) JTISOW

HW103-00082286

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 19 of 36 PageID #:

4081
U . S . Patent Oct . 24 , 2017 Sheet 17 of 23 US 9 , 800 , 708 B2 

Google - hotel at barcelona 

Web Maps 

Alice 

060 - 524 - 3849 ( at Spain ) 

+ 34 - 60 - 524 Phone Call 

060 - 524 - 3849 

Welcome to PU 

PPC creates a new mobile experience 

h 801 

FIG . 17 

HW103-00082287

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 20 of 36 PageID #:

4082
U . S . Patent Oct . 24 , 2017 Sheet 18 of 23 US 9 , 800 , 708 B2 

813 

Google 

hotel at barcelona 

Web Maps 

Alice - 811 

060 - 524 - 3849 ( at Spain ) 

+ 34 - 60 - 524 - 3849 ( at Korea ) 

0605243849 

Now Calling . . . 

Welcome to Phone PC Convergence Solution 

PPC creates a new mobile experience + Add phone Number 

BO 

FIG . 18 

|

S

HW103-00082288

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 21 of 36 PageID #:

4083
U . S . Patent Oct . 24 , 2017 Sheet 19 of 23 US 9 , 800 , 708 B2 

823 

PPC Solution Demo Page - Microsoft Internet Explorer CON 

11 111011 

C : WClipping WPPC English Demo . htm 

SMS 

SMS 

gmlee7 @ gmail . com | My Notebooks | Web History 

Google 

hotel at barcelona Search 

Web Maps 

Alice 

+ 34 - 60 - 524 - 3849 ( at Korea ) 

: Welcome to Phone PC Convergence Solution 

PPC . creates a new mobile experience 

FIG . 19A 

(

|

|

821

HW103-00082289

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 22 of 36 PageID #:

4084
atent Oct . 24 , 2017 Sheet 20 of 23 US 9 , 800 , 708 B2 

825 

Google 

hotel at barcelona 

Web Maps 

Alice 

060 - 524 - 3849 ( at Spain ) 

+ 34 - 60 - 524 - 3849 ( at Korea ) 

SMS Sender 

? Content O Send List 

Alice ( 0112345678 

060 - 524 - 3849 0112345678 ( at Spain ) 

+ 34 - 60 - 524 - 3849 ( at Korea ) 

SEND CANCEL 

Phone Book 

All | [ frequence ) | Recent ) | Group 

Welcome to Phone PC Converge 

PPC creates a new mobile 

Contact Point Gene 

010 - 7339 - 1126 ( at Korea ) 

+ 82 - 10 - 7339 - 1126 ( st Spain ) 

NOT FOUND 

FIG . 19B 

SMS Sender 

TO : 011 - 234 - 5678 V 

Alice 

+ 34 - 60 - 524 - 3849 

Welcome to Phone PC 

Convergence Solution ! 

PPC creates a new mobile experience 

Send Cancel 

FIG . 19C 

U.S.

:

)

+)

|

[ | C )

Yul Col

|

HW103-00082290

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 23 of 36 PageID #:

4085
U . S . Patent Oct . 24 , 2017 Sheet 21 of 23 US 9 , 800 , 708 B2 

Local business results for hotel at near Barcelona , Spain 

A . Korea - > Offange ( SKT Roaming ) - 001 

B . Korea - > Teletonica ( KTF Roaming ) - 0 

IC , Korea Boarning - Korea - 010 - 7339 

10 . Hotel 18984 - + 34 935 529552 - 481 

E . KITE . Roaming - > Orange - + 34 - 60 - 5 

F . SKT Boarming - > Teletonica - - 34 - 60 | G . Casa Camper - A member of Design 

H . HOTEL RESINA BARCELONA - WWW 

1 . Hotel Arts Bercelona - WWW . Fitzcarlto 

J . Hotel . Regencia Colon 3 - WWW .hotel - 

- 

- 

. . . , . . ... . . . . . . . . . . . . . . . . . . . . . . . . . f + . . . . . . . . . . . 

II . . . . . . . . 

FIG . 20A 

Z|

HW103-00082291

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 24 of 36 PageID #:

4086
atent Oct . 24 , 2017 Sheet 22 of 23 US 9 , 800 , 708 B2 

Phone Memo 

A . Korea 

B . Korea 

C . Korea 

D . Hotel 

E . KTF 

F . SKT 

G . Casa 

H . HOT 

1 . Hotel 

J . Hotel 

2009 / 02 / 25 11 : 06 : 43 

W 833 

[ Time 

file : / / C : WDocuments and 

Key map 

Send Cancel 

FIG . 20B 

Memo Viewer 

A . Korea 

B . Korea 

C . Korea 

D . Hotel 

E . KTF 

F . SKT 

G . Casa 

H . HOT 

1 . Hotel 

J . Hotel 

Time : 2009 / 02 / 25 11 : 07 : 50 

Keywords : map 

OK OK 

FIG . 20C 

U.S.

Tar

C

G

Ue

|

|

|

Full ce

Tan

1

WhB

P,

HW103-00082292

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 25 of 36 PageID #:

4087
U . S . Patent Oct . 24 , 2017 Sheet 23 of 23 US 9 , 800 , 708 B2 

wowa wote wa c wato WebA Web B Web C Web D 

Internet Explorer AllX 

Web A . 

FIG . 21 

HW103-00082293

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 26 of 36 PageID #:

4088
US 9 , 800 , 708 B2 

OPERATION METHOD AND SYSTEM OF this case , as in SMS , the user uses the system after sub 

MOBILE TERMINAL scribing to the separate service system providing MMS , without using his / her mobile terminal . Also , the user may 

PRIORITY download desired images to the mobile terminal and then 

5 send them using an MMS function of the mobile terminal . This application is a Continuation application of U . S . In order to send text to other mobile terminals , the user 

patent application Ser . No . 14 / 252 , 310 , filed in the U . S . can use a separate messaging service that is accessible via a 

Patent and Trademark Office ( USPTO ) on Apr . 14 , 2014 , computer . The user is usually required to subscribe to the 

which claims priority under 35 U . S . C . $ 120 to U . S . patent separate messaging service . application Ser . No . 12 / 706 , 591 , filed in the USPTO on Feb . 10 If a user sends messages to other mobile terminals over 

16 , 2010 , and is now issued as U . S . Pat . No . 8 , 725 , 209 on his / her own mobile terminal without using the separate 

May 13 , 2014 , which claims priority under 35 U . S . C . messaging service accessible via a computer , the user 

$ 119 ( a ) to a Korean Patent Application filed in the Korean directly inputs the content displayed on a computer screen to 

Intellectual Property Office on Feb . 13 , 2009 and assigned the mobile terminal using a keypad of the terminal , causing 

Serial No . 10 - 2009 - 0012155 , and to a Korean Patent Appli - 15 inconvenience . cation filed in the Korean Intellectual Property Office on A more complex process is required when a user , who is 

Mar . 31 , 2009 and assigned Serial No . 10 - 2009 - 0027864 , using a computer , delivers images on some areas of a 

the entire disclosure of each of which is incorporated herein computer screen to his / her own mobile terminal , or trans 

by reference . mits the images to other mobile terminals through his / her 

20 own mobile terminal . BACKGROUND OF THE INVENTION First , an image to be transmitted among the data displayed 

on a computer screen is stored in the computer as a particular 1 . Field of the Invention image file . The computer and a mobile terminal are con 

The present invention relates generally to a mobile com nected in a wired or wireless manner , and the stored image munication terminal ( or “ mobile terminal ” ) , and more par - 25 file is downloaded to the mobile terminal . Thereafter , if it is 

ticularly , to a method and system in which a mobile terminal desired to send a message by MMS , the mobile terminal 

connected to an arbitrary electronic device interacts with the executes an MMS function , attaches the downloaded image arbitrary electronic device . file to the message , inputs a phone number of a receiving 2 . Description of the Related Art mobile terminal , and sends the message . With the general use of mobile terminals and the devel - 30 In this case , due to the complex procedure , only a person 

opment of their functions , applications which have been with a certain level of knowledge about a file download 

implemented in electronic devices such as a computer , a method between the computer and the mobile terminal can 

television and a Portable Multimedia Player ( PMP ) are now store a particular image file in his / her mobile terminal . available even in mobile terminals , and functional applica - Besides , the user needs to directly input a phone number 

tions of mobile terminals can also be executed in a computer . 35 to the mobile terminal even when dialing a phone number 

In addition , there is an increase in number of applications displayed on the computer screen or sending an SMS 

which are executed through interaction between a mobile message and an MMS message to the phone number . terminal and a computer . In the above examples , the user is required to input many For example , such applications may include an applica - characters using a small keypad on the mobile terminal , tion for transmitting various information stored in a com - 40 causing inconvenience , and there is a possibility that char 

puter , such as contact information , schedule information , acters or phone numbers may be wrongly input . In addition , photo files , music files , video files and Internet bookmark the user must check both the computer screen and the mobile 

information , to a mobile terminal , and an application for terminal screen . downloading images and the like stored in a mobile terminal If a user wants to output an image file stored in a mobile 

into a computer . For execution of these applications , the 45 terminal using a printer connected to a computer , the user 

computer and the mobile terminal should be connected to needs to first download the image file from the mobile 

each other in a wired or wireless manner , and synchroniza - terminal to the computer . Then the user needs to run an 

tion may be achieved to keep the latest information between application required to print an image file by manipulating the two devices . Also , there is an application for delivering the computer , search for the image file , and make a request 

digital multimedia broadcast data received in a mobile 50 to print the image file through the printer . That is , the user 

terminal to a connected television to display the received must manipulate both the mobile terminal and the computer 

broadcast data on the television . in order to print image files stored in the mobile terminal . As another example , there is an application , with which a 

computer sends messages based on Short Message Service SUMMARY OF THE INVENTION 

( SMS ) and Multimedia Message Service ( MMS ) which are 55 

functions of mobile terminals . Accordingly , a user may send An aspect of the present invention is to address at least the 

SMS messages or MMS messages from a computer con - above - mentioned problems and / or disadvantages and to 

nected to the Internet to mobile terminals of other users . provide at least the advantages described below . Accord 

Since the user sends messages based on SMS provided by a ingly , an aspect of the present invention provides a method 

separate external service system without using his / her own 60 and system in which a user may operate functions of a 

mobile terminal , it is common that the user should subscribe mobile terminal without directly manipulating the mobile 

to the SMS service before sending the messages . terminal . MMS generally transmits data generated by and / or stored Another aspect of the present invention provides a method 

in a mobile terminal , for example , photo files created by and system in which , in a mobile terminal , a user may easily 

photographing with a mobile terminal . However , in some 65 use content displayed on and / or stored in an electronic 

cases , a user , who is using a computer , may send images device that is connected to the mobile terminal in a wired or 

stored in the computer to mobile terminals of other users . In wireless manner . 

1 2

HW103-00082294

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 27 of 36 PageID #:

4089
US 9 , 800 , 708 B2 

Ure tructures . 

A further another aspect of the present invention provides FIGS . 17 , 18 , 19A - 19C , 20A - 20C , and 21 are diagrams 

a method and system in which a user executes an operation illustrating screens displayed on a computer and a mobile 

associated with a function of a mobile terminal in a separate terminal according to various embodiments of the present electronic device by manipulating only the mobile terminal invention . in the state where the electronic device and the mobile 5 Throughout the drawings , the same drawing reference 

terminal are connected . numerals will be understood to refer to the same elements , Yet another aspect of the present invention provides a 

method and system in which an event that has occurred in a 

mobile terminal may be executed in a separate electronic DETAILED DESCRIPTION OF EXEMPLARY 

device connected to the mobile terminal . 10 EMBODIMENTS 

In accordance with one aspect of the present invention , an 

operation method of a mobile terminal connected to an The following description with reference to the accom electronic device is provided . The operation method panying drawings is provided to assist in a comprehensive includes extracting a piece of content displayed on a portion understanding of exemplary embodiments of the invention of a display of the electronic device , corresponding to a user 15 as defined by the claims and their equivalents . It includes selection , from among multiple pieces of content displayed various specific details to assist in that understanding but on the display ; displaying a function of the mobile terminal these are to be regarded as merely exemplary . Accordingly , corresponding to the extracted piece of content ; and deliv 

ering , to the mobile terminal , the extracted piece of content those of ordinary skill in the art will recognize that various 

and a command of the function of the mobile terminal that 20 changes and modifications of the embodiments described 

will execute the extracted piece of content . herein can be made without departing from the scope and 

In accordance with another aspect of the present inven - spirit of the invention . In addition , descriptions of well 

tion , an electronic device connected to a mobile terminal is known functions and constructions are omitted for clarity provided . The electronic device includes a display config - and conciseness . ured to display content ; and a processor configured to extract 25 According to embodiments of the present invention , in the 

a piece of content displayed on a portion of the display , state where a separate electronic device and a mobile ter 

corresponding to a user selection , from among multiple minal are connected , a user may operate functions of the 

pieces of content displayed on the display , display a function mobile terminal by manipulating only the electronic device 

of the mobile terminal corresponding to the extracted piece without manipulating the mobile terminal , or execute opera 

of content , and deliver to the mobile terminal the extracted 30 tions associated with functions of the mobile terminal in the 

piece of content and a command of the function of the electronic device by manipulating only the mobile terminal . mobile terminal that will execute the extracted piece of The electronic device refers to a device that can enable 

content . data processing , execute at least one application indepen 

In accordance with another aspect of the present inven - dently , and display or store the data generated by the 

tion , a non - transitory computer readable storage medium 35 application execution , and the electronic device includes a 

that stores a program including a command of an operation minimum user input unit . Such electronic devices include 

method of an electronic device is provided . The non - tran - computers , televisions , PMPs , Personal Digital Assistants 

sitory computer readable storage medium includes a first ( PDA ) , electronic dictionaries , mobile terminals , digital 

command set that extracts a piece of content displayed on a multimedia receivers , electronic book terminals , etc . portion of a display of the electronic device , corresponding 40 According to the present invention , an electronic device 

to a user selection , from among multiple pieces of content may deliver content and a functional application command 

displayed on the display ; a second command set that dis - of a mobile terminal that will execute the content , which are 

plays a function of the mobile terminal corresponding to the selected by a user , to the mobile terminal , and the mobile 

extracted piece of content ; and a third command set that terminal may execute the functional application using the 

delivers , to the mobile terminal , the extracted piece of 45 provided content . If a user selects arbitrary content from 

content and a command of the function of the mobile among content displayed on and / or stored in the electronic 

terminal that will execute the extracted piece of content . device , and selects a functional application and its associ 

ated command of the mobile terminal that will execute the 

BRIEF DESCRIPTION OF THE DRAWINGS content , then the electronic device delivers them to the 

50 mobile terminal and the mobile terminal executes the func 

The above and other aspects , features and advantages of tional application corresponding to the command using the 

certain exemplary embodiments of the present invention will received content . be more apparent from the following description taken in An event that has occurred in a mobile terminal may be 

conjunction with the accompanying drawings , in which : delivered to an electronic device connected to the mobile 

FIG . 1 is a diagram illustrating a system configuration 55 terminal so that a user may respond to the event using the 

according to an embodiment of the present invention ; electronic device . An operation for the response may be 

FIG . 2 is a diagram illustrating configurations of a com - executed in the mobile terminal or in both the mobile 

puter and a mobile terminal according to an embodiment of terminal and the electronic device in an interactive manner . the present invention ; If the user delivers a particular command to the connected 

FIG . 3 is a flowchart illustrating an operation of a com - 60 electronic device by manipulating the mobile terminal , the 

puter according to an embodiment of the present invention ; electronic device may execute an application associated with 

FIG . 4 is a flowchart illustrating an operation of a mobile a function of the mobile terminal according to the command . terminal according to an embodiment of the present inven An example of a system to which the present invention is 

tion ; applicable is shown in FIG . 1 , in which a computer 100 , i . e . FIGS . 5 to 16 are flow diagrams illustrating operations of 65 an electronic device , is connected to a mobile terminal 200 . a computer and a mobile terminal according to various The computer 100 includes an interactive service appli 

embodiments of the present invention ; and cation 140 , a content extractor 130 , a phone service appli 

3 4

and

HW103-00082295

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 28 of 36 PageID #:

4090
US 9 , 800 , 708 B2 

cation 120 , a device agent 110 , a user input device ( not provide at least one of call sending / receiving , SMS sending , shown ) , and a display device ( not shown ) . and MMS sending items . If execution of a particular func 

The user input device is a device by which a user may tional application is granted or selected by the user , the 

input data , and may include , for example , a keyboard , a content extractor 130 further includes information about the 

mouse , etc . 5 functional application in the execution data and delivers the The interactive service application 140 is an application execution data to the phone service application 120 . for allowing the computer 100 and the mobile terminal 200 Also , even if the user searches for and selects a data file to execute an operation associated with a function of the stored in the computer 100 by means of the search menu mobile terminal 200 in an interactive manner . The interac provided in the context menu without using the select tive service application 140 sets an interactive service mode 10 indicator , the content extractor 130 may provide at least one during its execution . In order to allow the mobile terminal functional application of the mobile terminal 200 , which can 200 to use content that is displayed on a screen by an 

use the selected data file . arbitrary application being executed in the computer 100 , or 

content such as a data file being used by a particular If the user selects a particular functional application by 

application , by means of multiple functional applications of 15 is means " of the context menu in a state where the user has not 

the mobile terminal 200 connected to the computer 100 , the selected any content using the select indicator or the search 

interactive service application 140 provides a select indica menu , the content extractor 130 may configure execution 

tor for allowing a user to select content and a context menu data including only information about the selected func 

tional application . Thereafter , the content extractor 130 may for allowing the user to select a functional application of the include in the execution data content that is selected or mobile terminal 200 . The select indicator may include , for 20 created according to a user input , for example , a select example , a cursor , a mouse pointer , a select window , and the 

like . The select window is an indicator for adjusting and indicator , a user ' s direct input , or a search for data stored in 

deciding a size of an image to be captured on a screen of the the computer 100 . The content inclusion may be achieved by the content extractor 130 or the phone service application computer 100 . The context menu includes a menu item 120 . corresponding to each of the multiple functional applications 25 The execution data may include information about a of the mobile terminal 200 , a menu item for data transmis 

sion / reception between the computer 100 and the mobile functional application to be executed in the mobile terminal 

terminal 200 , and a search menu capable of searching for 200 , information needed for execution of the functional 

various data files stored in the computer 100 . The menu application , and content selected by the user . The execution 

items may include menu items corresponding to , for 30 data may be configured as shown in Table 1 according to an 

embodiment of the present invention . example , call sending , SMS sending / receiving , MMS send 

ing / receiving , memo writing , etc . and the various data files 

may include an image file , a music file , a video file , a text TABLE 1 

file , and the like . This context menu may be provided in the Item Length Example form of a tool bar . 35 – Further , the interactive service application 140 invokes Type Type 1 line SMS sending 

MMS sending the content extractor 130 during its execution , and invokes Memo writing / reading the phone service application 120 according to a user input . Call sending The content extractor 130 extracts data that the user Name 1 line John Doe 

selected using the select indicator , from among data that is 40 Phone Number 1 line 010 - 1234 - 5678 

Attached File 1 line 20090123134400328 . jpg displayed on the screen as an arbitrary application is Date 1 line 20090123134400 executed in the computer 100 , and from a data file being ( 13 : 44 : 00 , Jan . 23 , 2009 ) used in an application being executed , and delivers the URL 1 line http : / / www . abcd . net / 

extracted data to the phone service application 120 after Tag , Keyword 1 line address , memo 

Content 1 line text data converting the extracted data into an appropriate format . 45 

For example , if a user selects text using a cursor or a 

mouse pointer in the state where a web browser is executed Referring to Table 1 , a Type field includes information 

and an arbitrary web page is displayed , the content extractor indicating a type and a command of a functional application 

130 extracts the selected text , configures execution data of a mobile terminal , which are selected by the user . A name 

including the extracted text , and delivers the configured 50 list of a receiving user is stored in a Name field , and a 

execution data to the phone service application 120 . The receiving phone number list is stored in a Phone Number 

content extractor 130 may also extract text or an image from field . An Attached File field may include content selected by 

a displayed text file in the state where a text application is the user . A time at which the execution data was configured 

executed . As another example , if the user selects a music file is stored in a Date field , and if content is extracted from a 

play application using a mouse pointer in the state where an 55 web browser , a relevant Uniform Resource Locator ( URL ) arbitrary music file is being played by the music file play address may be stored in a URL field . Separate data input 

application , the content extractor 130 may search the from the user is stored in a Tag , Keyword field , and content 

memory of the computer 100 for the music file being played , in text form , selected by the user , may be stored in a Content 

extract the searched music file from the memory , and field . configure execution data including the extracted music file . 60 In the execution data , a field including the content 

If the content extraction is completed , the content extrac - selected by the user is not fixedly defined , and the selected 

tor 130 may provide at least one functional application item content may be included in an appropriate field according to 

of the mobile terminal 200 , which can use the extracted a type of the content . For example , the content is included 

content , according to a type of the extracted content . As in in the Phone Number field if its type is a phone number , and 

the example above , if text is selected , the content extractor 65 the content is included in the Content field if its type is a 

130 determines whether the selected text is a phone number general string . In addition , the content will be included in the 

consisting of a number . If so , the content extractor 130 may Attached File field if its type is an image file or a music file . 

5 6

HW103-00082296

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 29 of 36 PageID #:

4091
US 9 , 800 , 708 B2 

Accordingly , only the information necessary for a type of a The SMS sender / receiver application 122 is activated if 

functional application to be executed will be stored in a SMS is designated as a type in a request of the interactive 

related field . service application 140 or in execution data provided from 

Other information except for a type and a command of a the content extractor 130 . The SMS sender / receiver appli functional application of a mobile terminal and an extracted 5 cation 122 may display an SMS sending user interface with 

content , which are stored in the Type field , may be expressed text included in the execution data displayed on the SMS 

as additional information of the application . sending user interface . The SMS sender / receiver application 

The phone service application 120 provides the user with 122 delivers execution data including an SMS send com 

a user interface corresponding to each functional application mand , an SMS body text and an SMS receiving number list 

of the mobile terminal 200 and a user interface for a data 10 to the device agent 110 in response to user input . The MMS sender / receiver application 123 is activated if transmission / reception function between the computer 100 MMS is designated as a type in a request of the interactive and the mobile terminal 200 , thereby inducing an additional service application 140 or in execution data provided from input of the user if needed , or providing the user with the content extractor 130 . The MMS sender / receiver appli information about the progress of a functional application 15 cation 123 may display an MMS sending user interface with 

being executed in interaction with the mobile terminal 200 . a text or an attached file included in the execution data Therefore , the phone service application 120 may provide a displayed on the MMS sending user interface . The MMS 

user interface for an associated functional application based sender / receiver application 123 delivers execution data 

on information included in the execution data provided from including an MMS send command , MMS content and an 

the content extractor 130 . If an end is input by the user , the 20 MMS receiving number list to the device agent 110 in 

phone service application 120 may deliver the execution response to user input . data with a functional application command to the device T he image capture application 124 is an application that 

agent 110 . is activated to capture an image on a screen of the computer 

The device agent 110 manages a connection between 100 with a select window . Upon receiving execution data in 

computer 100 and the mobile terminal 200 , processes execu - 25 which a captured image file is included and a memo appli tion data received from the phone service application 120 , cation is designated as a type , the image capture application 

and delivers the execution data to a phone agent 210 in the 124 delivers the received execution data to the memo writer 

mobile terminal 200 in interaction with the phone agent 210 application 126 . The memo writer application 126 displays 

in the mobile terminal 200 . The device agent 110 may a user interface on which a captured image , related URL 

download a phone book or phone number database stored in 30 information , time information and the like are displayed and 

the mobile terminal 200 , and manage the phone book . which requires the user to enter a keyword . The memo writer 

While the computer 100 serves as an example of an application 126 delivers execution data including a memo 

electronic device in FIG . 1 , the interactive service applica - write command , a captured image , an associated URL , time 

tion 140 , the content extractor 130 , the phone service information , etc . to the device agent 110 based on user input . application 120 and the device agent 110 may operate in a 35 The memo viewer application 125 fetches files stored in 

similar manner regardless of the electronic device in which the mobile terminal 200 via the FTP client 127 and displays they are installed . Although a computer will be designated as the files . an example of the electronic device in the following descrip - The device agent 110 may include a connection manager 

tion , the electronic device shall not be limited to a computer . 112 and a phone book manager 111 . The connection man 

A configuration of the computer 100 according to an 40 ager 112 manages a connection between the computer 100 

embodiment of the present invention is shown in FIG . 2 . and the mobile terminal 200 in interaction with a connection 

Referring to FIG . 2 , the interactive service application 140 manager 211 in the mobile terminal 200 . The phone book 

in the computer 100 includes a tool bar 141 and a context manager 111 downloads a phone book or phone number 

menu 142 , which provide the above - described menus pro - database stored in the mobile terminal 200 , and synchro vided in the context menu and a menu capable of designating 45 nizes phone book information stored in the computer 100 

a type of a select indicator . with phone book information stored in the mobile terminal 

The phone service application 120 may include a call 200 . sender / receiver application 121 , an SMS sender / receiver Turning back to FIG . 1 , the mobile terminal 200 includes 

application 122 , an MMS sender / receiver application 123 , the phone agent 210 , a phone application 220 , a user input 

an image capture application 124 , a memo viewer applica - 50 unit not shown ) , a display unit not shown ) , a Radio 

tion 125 , a memo writer application 126 , and an File Frequency ( RF ) unit ( not shown ) , and a memory unit ( not 

Transfer Protocol ( FTP ) client 127 , which correspond to shown ) according to an embodiment of the present inven 

respective functional applications of the mobile terminal tion . The user input unit may include a keypad , a touch pad , 200 . Although not shown in the drawings , the phone service or the like , and the RF unit is responsible for transmission 

application 120 may include various other functional appli - 55 reception and processing of radio signals . The memory unit 

cations executed in the mobile terminal 200 , including , for stores various program data required to operate and control 

example , a video call application , a music file play appli the mobile terminal 200 , reference data , updatable archival 

cation , and a camera application . data , etc . and the memory unit is provided as a working 

The call sender / receiver application 121 is activated if call memory . sending / receiving is designated as a type in a request of the 60 The phone application 220 includes various functional 

interactive service application 140 or in execution data applications corresponding to respective functions to 

provided from the content extractor 130 . The call sender / execute a variety of functions provided in the mobile ter 

receiver application 121 may provide a user interface that minal 200 , and also includes an application for exchanging sends a call to a phone number carried on the execution data data with the computer 100 . or a phone number corresponding to a name , and may 65 The phone agent 210 performs communication with the 

deliver execution data with a call send command to the device agent 110 in the computer 100 in a wired or wireless 

device agent 110 . manner and delivers execution data received from the device 

7 8

HW103-00082297

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 30 of 36 PageID #:

4092
US 9 , 800 , 708 B2 

10 

agent 110 to the phone application 220 so that a proper the select indicator . For example , in a state where a web page functional application may be executed by the phone appli - is displayed as a web browser is executed , the user may cation 220 . select a particular number string by dragging a number 

That is , the phone application 220 executes a functional string in the web page using a mouse pointer in step 11 . Then 

application designated by execution data provided from the 5 the content extractor 130 automatically extracts the content 

phone agent 210 , using particular content included in the selected by the user , i . e . , the number string , and identifies its 

execution data . For example , upon receiving execution data type in step 13 . Depending on the type of the content , the 

with a call send command designated in the execution data , content extractor 130 provides a functional application item 

the phone application 220 sends a call to a phone number of the mobile terminal 200 capable of using the content in 

included in the execution data . 10 step 15 . If the content is a number string as in the above 

The phone agent 210 manages a wire / wireless connection example , menu items including call sending , SMS sending between the mobile terminal 200 and the computer 100 . The and MMS sending may be provided . The user selects an 

wire connection may include a connection via a data cable , application corresponding to any one of these menu items in 

and the wireless connection may include a Wireless Local step 17 . In response , the content extractor 130 configures 

Area Network ( WLAN ) . In accordance with an embodiment 15 execution data that corresponds to selected function and 

of the present invention , the phone agent 210 manages a command and includes the extracted content in interaction 

phone book or phone number database stored in the mobile with the phone service application 120 , and delivers the 

terminal 200 and shares information about the phone book execution data to the mobile terminal 200 via the device 

with the computer 100 . agent 110 in step 19 . An example of the configuration of the mobile terminal 20 Then the mobile terminal 200 receives the execution data 

200 is shown in FIG . 2 . Referring to FIG . 2 , the phone agent via the phone agent 210 in an interactive service mode in 

210 may include the connection manager 211 and a phone step 31 . The phone agent 210 identifies a functional appli 

book manager 212 . The connection manager 211 manages a cation and a command of the mobile terminal 200 , which 

connection between the computer 100 and the mobile ter - correspond to the execution data , and delivers the functional 

minal 200 in interaction with the connection manager 112 in 25 application and command to the phone application 220 in 

the computer 100 . The phone book manager 212 manages a step 33 . The phone application 220 executes the functional 

phone book or phone number database stored in the mobile application and command using the content included in the 

terminal 200 , and shares information about the phone book execution data in step 35 . with the computer 100 . For example , upon receiving execution data in which Call 

The phone application 220 in the mobile terminal 200 30 Sending is designated and which includes a phone number , may include an FTP server 221 , a call sender / receiver the mobile terminal 200 performs a call to the phone application 222 , an SMS sender / receiver application 223 , an number . MMS sender / receiver application 224 , and a memo viewer W hile it has been described in the above example that the 

application 225 . Although not shown in the drawing , the user first selects content and then selects its associated 

phone application 220 may include various other functional 35 functional application , the user may first select a functional 

applications executed in the mobile terminal 200 , including , application and then select content . In this case , if the user 

for example , a video call application , a music file play selects a particular functional application by means of the 

application , and a camera application . tool bar 141 or the context menu 142 , the content extractor 

Each of the applications performs a proper operation 130 configures execution data including information about 

according to a functional application designated by execu - 40 the particular functional application and delivers the execu 

tion data received from the phone agent 210 . tion data to the phone service application 120 . In response , For example , the call sender / receiver application 222 the phone service application 120 provides a user interface 

sends a call to a phone number provided from the phone for the functional application . Thereafter , content selected or 

agent 210 . The SMS sender / receiver application 223 sends input by the user shall be included in the execution data . text provided from the phone agent 210 and an SMS 45 A phone event , such as call receiving , message receiving 

message to each phone number in an SMS receiving number and alarming , may occur in the mobile terminal 200 . Opera 

list , and displays a user interface for monitoring the opera - tions of the mobile terminal 200 and the computer 100 in this 

tions . The MMS sender / receiver application 224 sends text case are shown in FIG . 5 . The phone event is an event that 

or an attached file provided from the phone agent 210 to occurs in the mobile terminal 200 with respect to an opera 

each phone number in an MMS receiving number list , and 50 tion of a functional application provided in the mobile 

displays a user interface for monitoring the operations . The terminal 200 . The phone event includes both an event 

memo viewer application 225 displays on a user interface occurring based on the data received externally , such as call 

memos provided from the phone agent 210 , including , for receiving and message receiving , and an event occurring 

example , a captured image , an associated URL , time infor - based on an internal operation , such as an alarm , power mation , and a keyword . A user may scan or search previ - 55 shortage alert and / or schedule alert . ously received memos with a search button on the user Referring to FIG . 5 , if a phone event occurs in the mobile 

interface . terminal 200 in step 71 , the phone application 220 detects 

With reference to FIGS . 3 and 4 , a description will be and delivers the phone event to the phone agent 210 , and the 

made of operations of the computer 100 and the mobile phone agent 210 delivers the phone event to the device agent 

terminal 200 according to an embodiment of the present 60 110 in step 73 . The device agent 110 delivers the phone 

invention in the state where the computer 100 and the mobile event received from the phone agent 210 to the phone 

terminal 200 are connected in a wired or wireless manner . service application 120 , and the phone service application If an interactive service mode is set in the computer 100 120 displays the phone event by activating a functional 

as a user runs the interactive service application 140 , the application associated with the received phone event in step interactive service application 140 in the computer 100 65 75 . Accordingly , the user may check the phone event and 

provides a select indicator . The user may select desired then input a response thereto to the computer 100 through a 

content among the data displayed on the computer 100 using user input device of the computer 100 . Upon detecting the 

m 

9

HW103-00082298

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 31 of 36 PageID #:

4093
US 9 , 800 , 708 B2 

12 

the 

response in step 77 , the phone service application 120 in the A process of terminating the connection between the 

computer 100 delivers the response to the mobile terminal phone agent 210 and the device agent 110 is shown in FIG . 200 and executes a functional application corresponding to 8 . Referring to FIG . 8 , the connection manager 112 in the 

the phone event in interaction with the mobile terminal 200 device agent 110 sends a Stop command to the connection 

in step 79 . 5 manager 211 in the phone agent 210 to request connection 

For example , in the case of a call receiving event , if the termination in step 451 . In response thereto , the connection 

user responds with the computer 100 , the computer 100 manager 211 in the phone agent 210 sends a Stop ACK to the 

sends a call receive response to the mobile terminal 200 , so connection manager 112 in the device agent 110 in step 453 , 

that a call channel is connected in the mobile terminal 200 . thereby terminating the connection . 10 Turning back to FIG . 6 , after the phone book synchroni The mobile terminal 200 delivers a voice signal received 10 zation is completed , a user 230 may select a web content over the call channel to the computer 100 so that the voice such as text , an image and a hyperlink on the content or web signal may be output through a speaker of the computer 100 . page displayed on a computer screen 240 in steps 305 and The computer 100 may deliver a voice signal received 307 . The content extractor 130 extracts the selected content through a microphone to the mobile terminal 200 so that the 15 such as a name , a phone number , a simple string , a URL and voice signal may be delivered to the other party through the an image in step 309 , and delivers data to each functional mobile terminal 200 . In other words , the user may respond application according to user preferences in step 311 . to a call received at the mobile terminal 200 and talk to the Accordingly , the phone service application 120 generates an caller , using the computer 100 . additional input according to user preferences or requests a While overall operations of the computer 100 and the 20 phone book , if necessary , to thereby receive a phone number mobile terminal 200 have been described so far , specific list and select a recipient , in interaction with the device agent embodiments of the present invention will be described with 110 in step 315 . After configuring execution data including reference to FIGS . 6 to 21 . functional application information and recipient information Referring to FIG . 6 , in the state where the computer 100 through this process , the service application 120 delivers the and the mobile terminal 200 are connected in a wired or 25 execution data to the phone agent 210 via the device agent wireless manner , if an interactive service mode is set as the 110 in steps 319 and 321 . The phone agent 210 identifies a interactive service application 140 is executed , the device functional application included in the received execution fun 

agent 110 sends a request for phone book data to the phone data , invokes the functional application , and delivers the agent 210 , and in response , the phone agent 210 delivers a content included in the execution data to the phone appli phone book and a call history to the device agent 110 , so that 30 cation 220 to execute SMS sending , MMS sending , memo phone book synchronization may be achieved between the storing , call sending , etc . in steps 323 and 325 . Thereafter , computer 100 and the mobile terminal 200 , in steps 301 and the phone application 220 generates an ACK and sends it up 303 . A detailed operation regarding this is shown in FIG . 7 . to the phone service application 120 via the phone agent 210 Referring to FIG . 7 , the connection manager 112 in the and the device agent 110 in step 327 . The results can be device agent 110 sends a Hello command to the connection n 35 displayed on a User Interface ( UI ) in step 329 . manager 211 in the phone agent 210 to request connection While it has been described that the above process check in step 401 . Upon receiving the Hello command , the extracts content in the computer 100 and delivers the content connection manager 211 in the phone agent 210 sends a to the mobile terminal 200 along with an application com Hello ACKnowledgement ( ACK ) to the device agent 210 in mand to be executed , the process of delivering a command step 403 . Thereafter , the connection manager 112 in the ne 40 associated with execution of a functional application from device agent 110 sends a request for a phone book to the the computer 100 to the mobile terminal 200 may be phone agent 210 in step 405 . The phone book manager 212 simplified as shown in FIG . 9 . Referring to FIG . 9 , the phone in the phone agent 210 delivers all address book information service application 120 configures execution data including or phone book information in the mobile terminal 200 to the a command for a functional application to be executed and device agent 110 via the connection manager 211 , and the 10 45 content , in interaction with the content extractor 130 , and device agent 110 stores the received phone book information delivers the execution data to the phone agent 210 via the in the phone book manager 111 in step 407 . The phone book device agent 110 in steps 501 and 503 . The phone agent 210 

manager 212 in the phone agent 210 delivers the last call log sends the content along with the command to the phone and the frequent call log in the mobile terminal 200 to the application 220 based on the received execution data in step device agent 110 via the connection manager 211 , and the so 50 505 . The phone application 220 executes or processes the device agent 110 stores the received phone book information provided command and content in step 507 , and sends an in the phone book manager 111 in steps 409 and 411 . Once ACK indicating the completed execution of the command up all the data is stored , the device agent 110 sends an ACK to to the phone service application 120 via the phone agent 210 the phone agent 210 . and the device agent 110 in step 509 . The execution of the In the above process , a data format of phone book 55 command and content may include , for example , call send information transmitted and received by the phone agent 210 ing , message sending , memo storing , and / or address book and the device agent 110 may be configured as shown in adding . Table 2 . FIGS . 10 and 17 are diagrams showing a process of 

automatically sending a call request to the mobile terminal TABLE 2 60 200 when a string is selected in a web page according to an 

Item Example embodiment of the present invention . As shown in FIG . 17 , in a state where a web page is 

Last Name Park displayed on the computer 100 , if a user input occurs that First Name Seri holds a cursor or a mouse pointer on a string for a prede Group Samsung Electronics 

Phone Number 010 - 234 - 5678 65 termined time , the content extractor 130 extracts the string 

" 060 - 524 - 3849 ” selected by the user . The content extractor 

130 determines that the extracted string is a phone number 

11

HW103-00082299

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 32 of 36 PageID #:

4094
14 

25 

US 9 , 800 , 708 B2 

13 

if the extracted string consists of numbers and special Since Call Sending is designated and only the name is 

characters used for expressing a phone number . For included in the execution data , the call sender / receiver 

example , since “ 060 - 524 - 3849 ” consists of numbers and the application 121 requests the phone book manager 111 in the 

special characters “ - ” , it may be determined to be a phone device agent 110 to search for a phone number using a name 

number . A type of the special characters indicating a phone 5 in step 603 . The phone book manager 111 searches for a 

number is predefined . If the extracted string is determined as phone number using the provided name , and delivers a 

a phone number , the content extractor 130 extracts only the number list to the call sender / receiver application 121 in step number string from the extracted string after removing the 605 . If there is no matching name in the phone book , the 

special characters . As shown in Table 3 , the content extractor phone book manager 111 sends a search failure message . 130 configures execution data that includes the extracted 10 The call sender / receiver application 121 displays the number 

phone number as a recipient number . A type of the execution list , so that the user may select a number or determine 

data is automatically designated as Call Sending . whether a correct number has been located . If Select or OK 

is input by the user in step 607 , the call sender / receiver 

TABLE 3 application 121 adds the selected phone number to the 

execution data shown in Table 4 and delivers the execution Item Length Content data to the device agent 110 to send a call request in step 

Type 1 line Call Sending 609 . At this time , the call sender / receiver application 121 

Name 1 line displays a call sending user interface 813 that indicates call 

Phone Number 1 line 0605243849 sending to the phone number as shown in FIG . 18 . The 

Attached File 1 line 20 device agent 110 delivers the received execution data to the Date 1 line 20090123134400 phone agent 210 to request call sending in step 611 , and the URL 1 line http : / / map . google . com / Tag , Keyword 1 line phone agent 210 delivers the received execution data to the 

Content 1 line call sender / receiver application 222 to request call sending 

in step 613 . In response , the call sender / receiver application 

222 sends a call to the phone number included in the As shown in FIG . 10 , the content extractor 130 delivers 

the execution data configured as shown in Table 3 to the call execution data , and sends a Call ACK indicating the execu 

sender / receiver application 121 in step 551 . Upon receiving tion of a call send command to the call sender / receiver 

the execution data , the call sender / receiver application 121 application 121 via the phone agent 210 and the device agent 

displays a call sending ( or phone call ) user interface 801 110 in step 615 . A call channel is formed if there is a 

indicating ‘ call connecting ' as shown in FIG . 17 , and deliv - * response from a recipient after the call is sent , and a voice 

ers the execution data to the connection manager 112 in the signal to be transmitted over the call channel is delivered to 

device agent 110 to request call sending in step 553 . The the computer 100 . Therefore , the user may perform a call on 

device agent 110 delivers the execution data to the phone the computer 100 . agent 210 to request call sending in step 555 . Upon receiving Next , a process of sending an SMS message will be 

the execution data shown in Table 3 , the phone agent 210 35 described with reference to FIGS . 12 and 19A to 19C . While 

delivers the execution data to the call sender / receiver appli - it has been described in the above embodiment that a cation 222 to request call sending in step 557 . In response , functional application associated with the content selected the call sender / receiver application 222 sends a call to the by the user is automatically provided , execution data may be phone number included in the execution data , and sends a 

Call ACK indicating the execution of a call send command 40 configured after the user selects content and a functional 

to the call sender / receiver application 121 via the phone 40 application to use . As shown in FIG . 19A , in the state where 

agent 210 and the device agent 110 in step 559 . A call a web page is displayed , if the user selects a string by 

channel is formed if there is a response from a recipient after dragging a mouse and selects an SMS sending context menu 

the call is sent , and a voice signal to be transmitted over the located in a tool bar 823 , the content extractor 130 extracts 

call channel is delivered to the computer 100 . Therefore , the the text selected by the user , configures execution data with 

user may perform a call on the computer 100 . the extracted text as shown in Table 5 , and delivers the With reference to FIGS . 11 and 18 , a description will be execution data to the SMS sender / receiver application 122 in 

made of an embodiment in which the string selected by the step 651 . user is not a phone number but a name . Referring to FIG . 18 , if a mouse pointer is located on a TABLE 5 

name " Alice ” 811 by the user , the content extractor 130 50 

extracts a string corresponding to the name “ Alice ” 811 , Item Length Content 

configures execution data with the extracted string as shown Type 1 line SMS Sending in Table 4 , and delivers the execution data to the call Name 1 line 

sender / receiver application 121 in step 601 . Phone Number 1 line 

55 Attached File 1 line Date 1 line TABLE 4 URL 1 line 

Tag , Keyword 1 line 

Item Length Content Content 1 line Alice 

+ 34 - 60 - 524 - 3849 Type 1 line Call Sending Welcome to Phone PC Convergence Solution ! 

Name 1 line Alice PPC creates a new mobile experience . Phone Number 1 line 

Attached File 1 line Date 1 line 20090123134400 Thereafter , as shown in FIG . 19B , the SMS sender / URL 1 line http : / / map . google . com / receiver application 122 displays an SMS sending user Tag , Keyword 1 line 

Content 1 line 65 interface 825 on which the selected text or content is 

displayed as the contents of an SMS message and by which 

the user may input a recipient phone number and additional 

60 

15

HW103-00082300

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 33 of 36 PageID #:

4095
US 9 , 800 , 708 B2 

15 

data . The user , as shown in FIG . 19B , may directly input a in FIG . 20A . The user selects an image area to be captured , recipient phone number to the displayed SMS sending user using the select window 831 . A size of the select window 

interface 825 , or may search for a desired phone number in 831 is adjustable according to user preferences . If the user 

phone book data stored in the phone book manager 111 . inputs OK , the image capture application 124 captures a 

Regarding the search in the phone book data , the user may 5 screen area designated by the user , stores it as an image file , search for a phone number by inputting a name or a number adds the image file to execution data shown in Table 6 , and 

into a data input area of “ Phone Book ” in the SMS sending delivers the execution data to the memo writer application 

user interface 825 of FIG . 19B , may select the entire address 126 in step 903 . The memo writer application 126 configures a memo book by selecting “ All ” , may select frequently called phone numbers by selecting “ Frequent ” , may select last called 7 10 writing user interface 833 according to the received execu 

tion data as shown in FIG . 20B , and displays the memo phone numbers by selecting “ Recent ” , or may select phone writing user interface 833 . The user inputs additional data on numbers in a particular group by selecting “ Group ” . the memo writing user interface 833 when necessary in step If there is a phone book search request from the user , the 905 . The memo writer application 126 adds the additional 

SM sender / receiver application 122 requests the phone 15 data to the execution data , and if the user selects a sending book manager 111 in the device agent 110 to search for a event on the memo writing user interface 833 , the memo number in step 653 . The phone book manager 111 searches writer application 126 delivers the execution data to the 

for phone numbers based on a received search option , and device agent 110 , thereby delivering memo information in delivers the searched phone numbers to the SMS sender step 907 . The device agent 110 sends the phone agent 210 a receiver application 122 in step 655 . The SMS sender / 20 control message indicating a plan to send memo write 

receiver application 122 displays the received phone num related data thereby to request memo writing in step 909 . bers on the SMS sending user interface 825 , adds the phone The phone agent 210 sends a Memo ACK indicating a ready number last selected by the user among the phone numbers to receive memo information to the device agent 110 in step 

to the execution data of Table 5 , and delivers the execution 911 . The device agent 110 delivers the execution data to the 

data to the device agent 110 to request SMS sending in step 25 phone agent 210 to transmit memo information in step 913 . 657 . The device agent 110 delivers the received execution The phone agent 210 delivers the stored memo data to the 

data to the phone agent 210 to request SMS sending in step memo viewer application 225 in step 915 , and the memo 

659 , and the phone agent 210 delivers the received execution viewer application 225 displays the memo or an image file 

data to the SMS sender / receiver application 223 to request on the screen of the mobile terminal 200 , which is shown in 

SMS sending in step 661 . In response , the SMS sender / 30 FIG . 20C . If the memo information transmission is com 

receiver application 223 sends an SMS message that pleted , the phone agent 210 sends an ACK indicating the 

includes text content included in the execution data , as a transmission completion to the device agent 110 in step 917 . phone number included in the execution data . At this time , While it has been described in the above example that the 

a screen of FIG . 19C may be displayed on the mobile captured image file is stored in the mobile terminal 200 by 

terminal 200 . Thereafter , the SMS sender / receiver applica - 35 memo writing , it is also possible to request to transmit the 

tion 223 sends an SMS ACK indicating the execution of an captured image file through an MMS message . SMS send command to the SMS sender / receiver application As another example , a process of delivering a command 

122 via the phone agent 210 and the device agent 110 in step from the mobile terminal 200 to the computer 100 will be 

described with reference to FIG . 14 . A process of sending an MMS message using a text 40 Referring to FIG . 14 , the phone application 220 delivers 

content is similar to the process of an SMS message except information or a command which has been input by means 

that execution data is designated as MMS sending . of a phone application , to the phone agent 210 in step 951 . As another example , a process of sending an image The phone agent 210 delivers the information to the device 

captured by the computer 100 to the mobile terminal 200 agent 110 in step 953 . The device agent 110 sends a 

will be described with reference to FIGS . 13 and 20A to 45 command to an appropriate phone service application 120 or 

20C . a computer application according to the nature of the infor 

In the state where a web page is displayed as shown in mation or command in step 955 . An example of the com 

FIG . 20A , if the user selects a memo write menu using a mand may include memo display , presentation control , context menu or a tool bar , the content extractor 130 extracts document output , etc . The phone service application 120 

a URL of the web page and a time , configures execution data 50 processes the information according to the command in step 

with the extracted data as shown in Table 6 , and delivers the 957 , and delivers the results to the mobile terminal 200 using execution data to the image capture application 124 in step an ACK in step 959 . As further another example , a process of displaying 

memo information stored in the mobile terminal 200 on the 

TABLE 6 55 computer 100 will be described with reference to FIGS . 15 

and 16 . FIG . 15 shows a process in which the FTP client 127 Item Length Content in the computer 100 and the FTP server 221 in the mobile 

Type 1 line Memo Writing terminal 200 directly exchange memo information accord 

Name 1 line ing to an embodiment of the present invention . FIG . 16 Phone Number 1 line 60 shows a process of exchanging memo information through Attached File 1 line 

Date 1 line 20090123134400 the device agent 110 and the phone agent 210 . URL 1 line http : / / map . google . com / Referring to FIG . 15 , if a user event occurs , the content 

Tag , Keyword 1 line extractor 130 executes the memo viewer application 125 in 

Content 1 line step 701 . The memo viewer application 125 executes the 

65 FTP client 127 in step 703 . Then the FTP client 127 sends 

The image capture application 124 provides a select a request for a memo file to the FTP server 221 in the mobile 

window 831 on the web page as a select indicator as shown terminal 200 in step 705 . In response , the FTP server 221 

663 . 

901 . 

16

HW103-00082301

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 34 of 36 PageID #:

4096
17 

extra 

US 9 , 800 , 708 B2 

18 

delivers a memo file to the FTP client 127 in step 707 , and wherein delivering the extracted piece of content and the 

the FTP client 127 provides a File Acquire ACK to the memo command of the function of the mobile terminal that 

viewer application 125 in step 709 . Then the memo viewer will execute the extracted piece of content comprises : 

application 125 displays the memo in step 711 . delivering , to the mobile terminal through a device agent The procedure of FIG . 16 is similar to that of FIG . 155 by the phone service application , the determined com 

except that the FTP client 127 sends a request for a memo mand of the function , the extracted piece of content , file to the FTP server 221 in the mobile terminal 200 through and additional information of the function . the connection manager 112 in the device agent 110 and the 3 . The operation method of claim 2 , wherein determining connection manager 211 in the phone agent 210 in steps 755 the command of the function of the mobile terminal that will 

and 757 , and then the FTP server 221 delivers a memo file execute the extracted piece of content , in response to the 

to the FTP client 127 via the connection manager 211 in the input from the user in interaction with the phone service 

phone agent 210 and the connection manager 112 in the application comprises : 

device agent 110 in step 761 . FIG . 21 shows a state where extracting the piece of content displayed on the portion of 

any one of several memos are selected and displayed in the 16 the display where the select indicator is located , in 

computer 100 . response to the input from the user ; 

As is apparent from the foregoing description , according providing at least one function of the mobile terminal , to embodiments of the present invention , a user may operate which corresponds to a type of the extracted piece of 

functions of a mobile terminal without directly manipulating content , in interaction with the phone service applica the mobile terminal . The user may easily use , in the mobile 20 tion ; 

terminal , the data displayed on a screen of or stored in a selecting a function of the mobile terminal corresponding 

separate electronic device like a computer that is connected to the input from the user from among the at least one 

to the mobile terminal in a wired or wireless manner . In function of the mobile terminal ; and addition , in a state where the electronic device and the determining the selected function of the mobile terminal 

mobile terminal are connected , the user may execute an 25 as a function that will execute the extracted piece of 

operation associated with a function of the mobile terminal content . in the electronic device by manipulating only the mobile 4 . The operation method of claim 2 , further comprising , terminal , so that an event that has occurred in the mobile upon receiving a phone event from the mobile terminal , terminal may be executed in the electronic device . displaying the phone event in interaction with the phone While the invention has been shown and described with 30 service application and delivering a user response to the 

reference to certain exemplary embodiments thereof , it will 

be understood by those skilled in the art that various changes 5 . The operation method of claim 2 , further comprising , in form and details may be made therein without departing upon receiving an application command from the mobile 

from the spirit and scope of the invention as defined by the terminal , executing an operation corresponding to the appli appended claims and their equivalents . 35 cation command . 6 . The operation method of claim 2 , further comprising What is claimed is : synchronizing , by the device agent and a phone agent of the 

1 . An operation method of an electronic device connected mobile terminal , address book information stored in the 

to a mobile terminal , the operation method comprising : mobile terminal with address book information stored in the 

extracting a piece of content displayed on a portion of a 40 electronic device during initial connection . display of the electronic device , corresponding to a user 7 . The operation method of claim 2 , further comprising : 

selection , from among multiple pieces of content dis if the extracted piece of content is a phone number , played on the display ; automatically delivering a call send command and a 

displaying a function of the mobile terminal correspond phone number to the mobile terminal through the 

ing to the extracted piece of content ; and 45 device agent . delivering , to the mobile terminal , the extracted piece of 8 . An electronic device connected to a mobile terminal , content and a command of the function of the mobile the electronic device comprising : 

terminal that will execute the extracted piece of con a display configured to display content ; and 

tent . a processor configured to : 

2 . The operation method of claim 1 , extract a piece of content displayed on a portion of the 

wherein extracting a piece of content displayed on the display , corresponding to a user selection , from 

portion of the display of the electronic device com among multiple pieces of content displayed on the 

prises : display , providing a select indicator capable of selecting content display a function of the mobile terminal corresponding that is displayed on the display of the electronic device 55 to the extracted piece of content , and 

as the electronic device executes an arbitrary applica deliver to the mobile terminal the extracted piece of 

tion ; and content and a command of the function of the mobile 

extracting the piece of content displayed on the portion of terminal that will execute the extracted piece of 

the display where the select indicator is located , in content . response to an input from a user , 60 9 . The electronic device of claim 8 , wherein the processor wherein displaying a function of the mobile terminal is further configured to : 

corresponding to the extracted piece of content com - provide a select indicator capable of selecting content that 

prises : is displayed on the display of the electronic device as 

determining a command of a function of the mobile the electronic device executes an arbitrary application ; 

terminal that will execute the extracted piece of con - 65 extract the piece of content displayed on the portion of the 

tent , in response to the input from the user in interaction display where the select indicator is located , in 

with a phone service application ; and response to an input from a user ; 

50 

mobile terminal.

HW103-00082302

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 35 of 36 PageID #:

4097
10 

US 9 , 800 , 708 B2 

20 

determine a command of a function of the mobile terminal automatically delivers a call send command and the phone that will execute the extracted piece of content , in number to the mobile terminal through the device agent . response to the input from the user in interaction with 15 . A non - transitory computer readable storage medium 

a phone service application ; and that stores a program including a command of an operation 

deliver the determined command of the function , the 5 method of an electronic device , the non - transitory computer extracted piece iece of of content content and and additional additional information information readable storage medium comprising : by the phone service application of the function to the a first command set that extracts a piece of content mobile terminal through a device agent . displayed on a portion of a display of the electronic 10 . The electronic device of claim 9 , wherein the proces device , corresponding to a user selection , from among sor is further configured to : multiple pieces of content displayed on the display ; extract the piece of content displayed on the portion of the 

a second command set that displays a function of the display where the select indicator is located , in 

response to the input from the user , mobile terminal corresponding to the extracted piece of 

content ; and provide at least one function of the mobile terminal , which corresponds to a type of the extracted piece of 15 16 a third command set that delivers , to the mobile terminal , a 

content , in interaction with the phone service applica the extracted piece of content and a command of the 

tion , function of the mobile terminal that will execute the 

select a function of the mobile terminal corresponding to extracted piece of content . 

the input from the user from among the at least one 16 . The non - transitory computer readable storage medium 

function of the mobile terminal , and 20 of claim 15 , further comprising : 

determine the selected function of the mobile terminal as a fourth command set that provides a select indicator 

a function that will execute the extracted piece of capable of selecting content that is displayed on the 

content . display of the electronic device as the electronic device 

11 . The electronic device of claim 10 , wherein upon executes an arbitrary application ; 

receiving a phone event from the mobile terminal , the 25 a fifth command set that extracts the piece of content 

processor is further configured to display the phone event in displayed on the portion of the display where the select 

interaction with the phone service application and to deliver indicator is located , in response to an input from a user ; 

a user response to the mobile terminal . a sixth command set that determines a command of a 12 . The electronic device of claim 10 , wherein upon function of the mobile terminal that will execute the receiving an application command from the mobile terminal , 30 extracted piece of content , in response to the input from the processor is further configured to execute an operation the user in interaction with a phone service application ; corresponding to the application command . and 13 . The electronic device of claim 10 , wherein the pro 

cessor and a phone agent of the mobile terminal synchronize a seventh command set that delivers , to the mobile 

address book information stored in the mobile terminal with 35 terminal through a device agent by the phone service 

address book information stored in the electronic device application , the determined command of the function , 

during initial connection . the extracted piece of content , and additional informa 

tion of the function . 14 . The electronic device of claim 10 , wherein if the 

extracted piece of content is a phone number , the processor * * * * * 

19

HW103-00082303

Case 2:23-cv-00103-JRG-RSP Document 94-12 Filed 05/30/24 Page 36 of 36 PageID #:

4098