Patent AU-2018253637-B2 (AU)
Title: Interactive tutorial integration
Inventor: CHANG WALTER W
IPC: G06Q10/00
Filed: 2018-10-29
Granted: 2021-07-29
Abstract: A tutorial for a given application may be leveraged to generate executable 
code that can then be executed within a native instruction service of the application.  
In this way, a software application may thus provide an integrated, interactive 
learning experience for a user, in a manner that extends beyond the instructional 
content included in the native instruction service, i.e., that includes at least a portion 
of the instructional content of the tutorial.
1/11 
100 
Computing device Memory104 Processor 106 
102 
Application 108 
Native Instruction Service 120 
Feature List 
Instruction 130 
Instruction Code Element 
122 Generator 
124 
Tutorial Manager 132 
Tutorial Annotator Annotated Feature Extractor 
134 Tutorial 146 
Content 
11381 
Tutorial Extractor Feature Mapper 
136 148 
PTFGenerator files Instruction Code 
140enrto 4 Generator 140 ---------------- _ 150 
Model Trainer 144 
User Device 112 
Display 114 
128 
Networkt126 
110 
Image 
116 
Tutorial 
1181 
FIG.1
2/11 
75s 
C ___O__O 1Li 
c0 
CL 000 DC 
o a~c-0oCc: 3 t 
0 oco )( 
- c - C c O~-0 o 0- e -- -- .o 
CNI ~Sa 0o-e o-ooo C 5D .0o90*~ 
O -O 0ooD C o oU -0 
-c LD .t ^1) --tE2 ee 
a) E ) c a (D - a 0 .  
0- (DU)U 
(U N 
0 CO 
Ch CD IIN 
-c 0 o o Nm m 
E A: tZ 1 
Ni < : 
0. (-) U-) F-C 
2 o a Eo 0= O3r > 0 
00 
y Mo t ooD 
c"oi orC$ E E CN 
24 0 
204o -5 
0 0 1: :3 =0Jw
3/11 
300 
Parse a tutorial file derived from a web page, the tutorial 
file representing a tutorial provided by the web page for a 
software application, and constructed in a hierarchical 
format in which tutorial attributes are specified with respect 
to corresponding values of the tutorial attributes, the values 302 including at least one instruction included in the web page 
for using the software application 
Execute a mapping of the tutorial attributes to application 
features of the software application 304 
Generate instructional code that is executable by a native 
instruction service of the software application to generate 
at least one instruction element within the software 
application, based on the mapping, the at least one 
instruction element identifying at least one application 
feature of the application features that is configured to 
enable completion of the at least one instruction 
FIG.3
4/11 
m U) 
a)) 
.0 a) 
a) 
a)-0 
0 I
-) -a -a) 
0 0 E 00o 
-Fu~)~ U) Fu ) U
5/11 
Conjunctive sequential one level steps 
• Sequential 
• Non-hierarchical 
S1 504 
AND (S1,S2,..,Sm) S 2  506 
Si 508 
502 Sk 510 
FIG.5 
Conjunctive sequential steps / substeps 
• Sequential 
• Non-hierarchical 
604 
606 
S,610 S1.1 608 
AND (S1 S2,..,Sm) 82612S11 
Si 
620 
60f2S S Si.1 614 
Step Si.2 616 
Sim6618 
FIG.6
6/11 
Disjunctive sequential step/substep 
• Sequential 
• Hierarchical 
704 
706 
- S1 
I,, 710 S 1.1 708 
-S2 Substep 
AND (Sl,S2,..,Sm 712 S S 
r- Si 
720 
702 Sk S.1 714 
Step Si.2 716 
Sim 718 
FIG.7 
Conjunctiveprocedure/step/substepstructure 
* Sequential 
• Hierarchical 
80480 
808 
P1  S1 814 S 810 
802 816 
AND (P1,P 2,..,Pm) ' 818 
822 S 
Pm, 
FIG.8
7/11 
Disjunctiveprocedure/step/substepstructure 
• Sequential 
• Hierarchical 
904 
K) ~90690 
AND (P1,P2 ,..,Pm) 9 4 910 
920 S2 0 902 916 S1.2 912 
4 918|S 
922 
Pm 
FIG.9
8/11 
1000 
1002 
El-root 1004 
header 1006 
-author: Rebecca Creger 4 
-author url: https://99design.com/blog/author/rebecca-creger/ 
--- categorytag list ,...- 1008 
--date: , 1010 1013 
-headertextlist-..-1012 
-- [0]: The Adjustment Layers in Photoshop are a group of a super useful,[ non-destructive image e 
[1]: With the adjustment layers, you can edit and discard your adjustments or restore your origi 
[2]: This will make your workflow in Photoshop more flexible and efficient, and is an absolute m 
[3]: For this guide we're going to go over the basics of each type of adjustment layer, working 
--imagebefore: https://99design-blog.imgix.net/blog/wp-content/uploads/2013/11/locate1.png 
procID:PSCreative-000001 -1016 *1014 
--title: Using Adjustment Layers .- 1018 
tutorialtype: tutorial .-.-1020 
--tutorial url: https://99designs.com/blog/design-tutorials/how-to-use-adobes-adjustment-layers-photo 
Sproc-list * ...- 1024 
n- [0] 1022 
--procname: Procedure 
--procoverviewtext: f--026 
-proc_rule: ANY 
0proc-steplist ,-1028 
[01] 
i--step_name:1 Brightness/Contrast-..-1030 
-steprule: Required -1032 103 
P-stepsentencelist -103435 
10] 
-stepsentenceaobjectcandidates 1036 
[0] 
ptfObjectConfidence:1 
ptfObject|D:_NoEmphlDbrigtness/contrast 
-ptfObjectName:Brigthness/Contrast 
--ptfObjectType:Tool 
[1] 1038 
ptfObjectConfidence: 0.75 
-ptfObjectlD: adjustAction 1030 
ptfObjectName: Adjust 
ptfObjectType: Action 
... stepsentencetext: ** Brightness/Contrast** makes adjustments to the tonal range of 
[1] 1032 
$-stepsentence-object-candidates kl 
stepsentencetext:The brightness slider is for adjusting the highlights in your im 
--stepurl: https://99design-blog.imagix.net/blog/wp-content/uploads/2013/11/BrightnessCont 
n [1]*1034 
FIG.10
9/11 
1100 
Generate markdown markup language representation of 
tutorial content 1102 
Determine structure levels of the tutorial content 1104 
Extract tutorial content and generate annotated, structured 
tutorial content 1106 
Classify annotated, structured tutorial content as a tutorial 
structure model 1108 
Generate PTF file for tutorial 1110 
FIG.11
10/11 
0J 
0 
z 0 
oU 0 )l 
U)U 
0 U E 
0 C)
0j 0 
co U) E a)
0 l U) co 0 0 
Co 0o 
0~ 00 U) SO~C 
0 n 0 -o 
0-a) a 0tc a) c )Fn0 C 
C- 0 az> ) 0 0C~ co co a 06 
L..J 
00 a a) a) 
0 Z ) 
o 0N 
a)) 
a'a))l 
(CN 0 5E, 
it 00.  
0~0
11/11 
1300 
Determine application features supported by native 
instruction service 1302 
Provide production function(s) for application features 1304 
1306 
Traverse tree structure of PTF 
Perform NLP to match PTF content with application feature 1308 
Select production function 1310 
Generate corresponding code segment 1312 
Add code snippet to native instruction service code 
hierarchy 1314 
1316 
Traversal No 
completed? 
1318 
Yes 
Generate executable native instruction service code 
FIG.13