（ＡＥＮ Project ２００５）

ALIVE:
AEN LMS ＆ Contents
Interoperability Validation
Experiment

March.2006
e-Learning Consortium Japan

3-１

3.Implementationof
informationexchange,etc.
regardinginteroperability problem, validation system, etc. in each AEN country
(AEN
LMS
＆
Contents
Interoperability
Validation
Experiment(hereinafter referred to as “ALIVE”)

3.1

Objective and target result

The validation experiment is to be performed of LMS and content products ofeach AEN country to check the SCORM 1.2 compatibility and interoperability. In
addition to the SCORM 2004 compatible products, this year also performs
testing of the SCORM 2004 compatible product. The activity in this year is thirdyear, and the ALIVE events have been held three times, following January andDecember 2004.
3.1.1 Objective(1) Holding of ALIVE 2005 (information exchange and interoperability experiment
between products)(2) Implementation of SCORM 1.2 standard compatible validation experiment(3) Implementation of SCORM 2004 standard compatible validation experiment3.1.2 Target result(1) Holding of “ALIVE 2005”
・Information exchange regarding each AEN country and interoperability, etc.
with ADL
・Implementation of interoperability experiment between each country products(2) Grasping of interoperability quality compatible to SCORM 1.2 and 2004
・Grasping of compatibility to SCORM standard
・Grasping of interoperability between each country products
・Grasping of SCORM compatibility of authoring tool preparation content(3) Check of test suite effectiveness, operability, etc.
(4) Preparation of compatibility check standard and supplemental operational
instruction of SCORM 2004 test suite

3.2

Activity overview

3.2.1 Participating body and participant
11 persons from 6 countries participated in the validation experiment.
Furthermore, 25 persons from 12 countries participated in ALIVE (International

3-２

Conference). Table 3-1 shows the participants of validation experiment.
Table 3-1
Country name

Name

Participant list

Organization, etc. belonged
ALEXIT

SCORM1.2 SCORM2004
Participation Participation

Korea

Mr. Park Choon Won

Malaysia

Mr. Muhammad Hasan Multimedia University○

Mr. Lim Kin Chew

Temasek Polytechnic○

Mr. Daniel Tan

Nanyang Technological University○

Viet Nam

Mr. Nguyen Anh Tuan

NCS

○

USA

Ms Brooks Jennifer○

Singapore

ADL○

Mr. Takeshi Kumazawa Human Science
Mr. Shingo Shibata
Japan

Secretariat

CompaqMr. Kiyoshi Nakabayashi NTT Resonant

○

○

○

○

Mr. Hiroshi Miyauchi

Sangyo Noritsu University○

Mr. Minoru Toida

SATT

○

Mr. Hiroyuki EndoRail

Mr. Kenichi Tsuji

NEC○Mr. Toshiyuki Kobayashi e-Leaning Consortium, JapanMr. Yosuke Yoshimura- ditto -Mr. Toshio Munemoto- ditto -3-３

Progress schedule

March

February

January

DecemberNovember

October

Septeme

August

July

April

Work item

Progress schedule

June

Table 3-2May

3 .2.2・Planning・Experiment by test suite
・Experiment between products▲

・ALIVE holding
・Preparation of compatibility
standard
・Activity summery

▲

3.2.3

Activity overview(1) Overall activity
Domestic committee was held to plan activity program and to implement
activity follow. The ALIVE International Conference was held in December toexchange information with each AEN country.
Concerning the interoperability validation experiment of each country product,
this year performed the SCORM 2004 compatible products in addition to
SCORM 1.2. The experiment methods were 2 types such as compatibility to
SCORM standard used ADL test suite and interoperability check between LMS
product of each country and content product. For the testing places, the testingused the test suite and the testing of interoperability experiment between
products were performed in each country and ALIVE conference room
respectively.
Concerning the SCORM 1.2 compatible products, the SCORM compatibility
testing for the contents provided by commercial authoring tool was performed.
The experiment of authoring tool experiment for the SCORM 2004 was not
performed due to no authoring tool product available.
In order to smoothly and steadily perform the compatibility inspection by thetest suite regarding the SCORM 2004, the material “SCORM 2004 test suite
guide”, etc. was provided.
(2) Domestic committee activity
Table 3-3
Conference
nameFirst precedent
conference
Second precedent
conference

Third precedent
conference

Domestic conference overview

Date held
November 2, 2005
November 10, 2005
November 29, 2005

Major agenda
・Review of activity program
・Overseas validation experiment・Validation experiment of authoring tool・Domestic validation experiment・Implementation of validation experiment
between domestic products・International conference agenda

3-４(3) International conference activity
An event called ALIVE was held in Tokyo in December 14 - 15, 2005 to
perform validation experiment, etc. between each country products. The details
are described in the sub-section 3.5.
3.3 Validation experiment of SCORM 1.2 standard compatible product3.3.1 Experiment overview(1) Experiment purpose and method
With use of the testing participating products, the interoperability inspectionused test suite and the interoperability experiment between LMS product andcontent product were performed. With use of commercial authoring tool thatdeclares the SCORM compliance, the compatibility experiment to the SCORM
1.2 standard was also performed.
Here describes the experiment performed with the test suite. The purposes ofthis testing are as follows:
・To check the SCORM standard compatibility of the testing participatingproducts,
・ To grasp the SCORM compatible level of the experiment participatingproducts, and・To check effectiveness and operability of the test suite.
The test suite used was Multilanguage compatible test suites (ADL test suiteVer. 1.2.7 and Multilanguage module) that were developed in last year, and thefollowing 2 functions were used:
・SCORM 1.2 LMS runtime environmental compatibility test
・SCORM 1.2 Content package compatibility test
Concerning the interoperability experiment between products, other country
content product was run with each country LMS product to test whether it
operates properly.
Concerning the authoring tool testing, the compatibility testing by the test
suite to the content created using the authoring tool was performed.
(2) Participating product
4 LMS products of 4 organizations from 4 countries were participated in theexperiment. Table 3-4 shows the participating body or organization, productname, etc. The authoring tool participating products are described in the
sub-section 3.3.4. In order to protect enterprise secret, a fictitious name wasused for experiment results, etc. without indicating the product name.

3-５Table 3-4

Participating body (or organization) and product list

Country
nameParticipating
Exmerimenter
body nameSingapore

Learning
Standards
Technical
Committee,
Singapore

LMS
Product name

Dr. Daniel Tan
MoodleMr. Lim Kin
Version 1.5.2
Chew

Content
Charactercode

Product nameCharacter
code

UTF-8

iT21,Ednovation-Biology,
VernierCalipers

UTF-8

SoftSimulator,
iLCBuilder

UTF-8

digital signal processing,e-Business

ascii

Content A (*1)

UTF-8

UTF-8
New Century
Mr. Tin Nguyen
Soft
Ba
Company

Trainware

MalaysiaMultimedia
University,
Malaysia

Mr.Muhammad
Hasan

MMLS
(Multimedia
Learning
System)Korea

ALEXIT

Mr. Park
Choon Won

LMS-A(*1)

Viet Nam

Total

UTF-8

UTF-8

4 products

4 products(*1) Since these were not scheduled for participating in experiment, they wereparticipated in only between products.
3.3.2

Compatibility experiment by test suite

As the results of experiment used the test suite, 1 failure/product among 3
products of LMS was detected. No failure was detected on 3 content products.
The non-conformed product ratio is 17% high (failure: 1 product/6 products). The
testing results and the failure detected are shown in Tables 3-5 /3-6 and Table
3-7 respectively.
Table 3-5

LMS experiment results by SCORM 1.2 test suite

Product nameCompatibility
Compatible level
Number of failure
occurredLMS product - A
x-

LMS product - B○
LMS-RTE2LMS product - C○
LMS-RTE2

1（F-01）

None

None

3-６

Table 3-6

Content experiment results by SCORM 1.2 test suite

Product nameCompatibility

Content Product - E

Content product - F

Content product - G○○
SCO-RTE1Mandatory○
SCO-RTE1Mandatory

None

None

Compatible level

SCO-RTE1

Number of failure
occurred

None

Table 3-7
Order No.

Phenomena, etc.
Alarm message for option data model was
displayed

F-01

3.3.3

Failure list by SCORM 1.2 test suite
Cause
SCORM standard
Infringement

Validation experiment between LMS and content products

In the experiment with 12 combinations of 4 LMS products and 4 content
products,

failure occurred in 2 combinations. The failure occurrence ratio is

17% high (2 failures/12 combinations).
Both 2 failures are out of SCORM standard, and failure relating to the SCORM
standards did not occur. Furthermore, for the product A, failure was detected in
testing by the test suite but not detected in the testing between products. From
this result, it can be assumed that the testing items with high inclusion to thestandard have been strictly performed. The experiment results and the failuresdetected are shown in Tables 3-8 and 3-9 respectively.
Table 3-8

Experiment results of interoperability between SCORM 1.2 productsContent

LMS

Product - A

Product - B

Product - C

Product - D

Product - E

-

○

○

○

Product - F

○

-

○

△ (F-02)

Product - G

○

○

-

○

Product - H○△(F-03）○-○ : run properly△ : run but failure occurred

x: did not run−: due to same body product, experiment was not performed

3-７

Table 3-9

Failure list of interoperability between SCORM 1.2 products

Order No.

3.3.4

Phenomena, etc.

F-02

Content is not suitably displayed

F-03

Failure occurred on manifest

Cause
Failure of out of
SCORM standard
Failure of encode
Failure of out of
SCORM standard

Compatibility experiment of authoring tool product3.3.4.1

Testing overview(1) Testing background and purpose
Authoring tool that declares the SCORM 1.2 standard compliance is sold inmarket. However, up to now day, third party institute did not have an
opportunity to check the SCORM standard compatibility and interoperability of
the authoring tools. The experiment is to check that the authoring tool complies
with the SCORM 1.2 standard and is the product took into account theinteroperability by the validation experiment.
(2) Testing method
Using the authoring tool for experiment, the content for inspection is toprovide based on specification (*1) designated. Using the Multilanguage
compatible test suite, the compatibility inspection of content for inspection
provided is to perform. Furthermore, it is to check whether the authoring tool has
been developed took into account “Application technology for improvement of
SCORM interoperability seen in the case study 1.2 edition” (*2).
*1 Specification that the SCORM function required for compatibility check
was specified*2 Specification for product validation that was provided by the e-Learning
Consortium, Japan
3.3.4.2

Experiment results

As the results of 5 products tested, failure of the interoperability in 1 product was
detected. Number of sample is small, however failure occurrence ratio is 20%
high. Since the SCORM compatibility of authoring tool can be comparatively andeasily checked with inspection used the test suite, further utilization of the test
suite is desired. Furthermore, if further improvement of the interoperability isplanned, establishment of the validation system of authoring tool will be needed.
For the product that failure was detected, the product was improved, resulting in
problem resolution after detection of failure. Table 3 -10 shows the experimentparticipating product. Since these are not scheduled for participating in testing,they were participated in only between products.
Table 3-10
No.

1
2
3
4
5

Experiment participating products of authoring tool

Authoring tool product nameAcademicWare Author (V4.0)
Epiplex (4.3 SP2)Internet Navigware material creation kit

LiveCreator (4)Xcalat Author V2.0

3-８

V8.0

VendorCompaqCybernet System
FujitsuRail
NTT Resonant

3.3.5

Questionnaire result to experiment participant

The results of questionnaire performed to the experiment participants this time
are shown in Table 3-11. This questionnaire was performed to the experimentparticipants of the SCORM 1.2 and the SCORM 2004.
For the meaning of the validation experiment, all bodies (organizations) repliedanswered to be meaningful, and continuous experiment is expected in future.
While for operability of the test suite, half of them has answered to be difficult touse, and improvement is expected. The utilization method, etc. of the SCORM
2004 test suite is seemed to be a delicate.
Table 3-11
No.

Questionnaire summery results of experiment participants

ItemAnswer column
・Was meaningful: 6 bodies
・Was not meaningful: 0

1

Meaning of this validation testing

2

Desires continuous implementation
・Desires: 6 bodies
of validation experiment after next
・Does not desire: 0
year

3

Effectiveness as SCOR standard
compatibility validation of test suite

・Effective: 6 bodies
・Not effective: 0

4

Easy use of test suite・Easy: 3 bodies
・Not easy: 2 bodies

5

Reason of “Not easy to” to use(multiple answers are possible)

・ Check item is many, and it takes a too long time forinspection (2 bodies)
・Due to English message display of test suite, it is difficult tounderstand (1 body)

6

Opinion and request of test suite(free entry)

・Extreme mental concentration was needed for conditionand operation of conformance test of SCORM 2004-LMS

Opinion to entire validation
experiment・Inclusion covers most of them, and it is very difficult to createsuch test specification by participant company
・It will better if test suite can be performed by separating test
item on every each theme
・Due to heavy processing during test suite tenting, frequent
freezes occurred. (SCORM 2004)
・Some cases do not display SCORM conformant even there is
not log error. (SCORM 2004)
・It will be better if Q and A sheet of case by case is available
・Wishes to perform testing for sequencing and navigation

7

3-９

3.3.6

Consideration of experiment result

3.3.6.1 Interoperability quality of each country product and effectiveness of test
suite(1) Interoperability quality of each country product is low
The SCORM testing product non-conformity ratio (number of non-conformity
product/number of product tested) that was found with the test suite is 17% high,the failure occurrence ratio between products (number of failure occurrence
combination/number of testing combination) that was found with theinteroperability experiment between products is also 17% high, and theinteroperability quality of each country product is low.
(2) Test suite is effective as SCORM compatibility inspection tool
2 SCORM standard infringed products were detected with the compatibility
testing used the test suite. Failure related to the SCORM standard was notdetected with the interoperability experiment between products. From this result,
the test suite is effective as the compatibility inspection tool.
(3) Grasping of SCORM compatible level
The SCORM compatible level has 3 steps of 1 – 3. The product corresponding
to the most-significant level 3 did not exist in these participating products. As forthe reason that does not support the most-significant level, problems such asthat LMS is development cost increase and that if the content is to be higher
level, the operable LMS is restricted, are considered.
The compatible level of LMS product that participated in this time is level 2 for
2 products, while the content is less than level 1.5 for 3 products, and problems
of interoperability in the compatible level does not occur between these products.
However, in the case the content more than compatible level 2 is implemented
with these LMS, problem of the operability may occur.
The following explains compatible level of LMS and content. As shown in
Table 3-12, in the case the LMS level is lower than the content level, problem
that a part of function of the content will not operate, may occur. Concept of thecompatible level is essence of the SCORM 1.2 and does not exist in the SCORM
2004. From this result, the SCORM 2004 is much better than the SCORM 1.2.

Table 3-12

Combination and interoperability of LMS and content level

Content
LMS
Level 1
Level 2
Level 3

LMS-R
TE1
LMS-R
TE2
LMS-R
TE3

Level 1

Level 1.5

Level 2

Level 3

SCO-RTE1＋Mandatory＋Optional＋Mandatory＋Optional

○

○

x

x

○

○

△

△

○

○

○

○

3-１０

○ : LMS covers all functions of contents that operate.

△: In the case the option that LMS uses content does not support, appropriate
function of content does not operate. Which option is supported or used by
LMS and content, depends on specification of each product.
×: Option function that content is using does not operate.
(4) Interoperability quality of authoring tool
As the results that 5 authoring tool products were performed, it was foundthat 1 product infringed the SCORM standard. The non-conformity ratio is
20% high, and it can be pointed out that the interoperability quality ofauthoring tool is low. It is desired that commercial authoring tool product thatdeclares the SCORM conformance should be checked the SCORM
conformance by the test suite.

3-１１

Validation experiment of SCORM 2004 standard compatible product

3.4

3.4.1

Experiment overview(1) Type and method of testing

Subjecttotheexperimentparticipatingproducts,

theinteroperability

inspection used the test suite and the interoperability experiment between thecontent products were performed. The test suite which uses for the experimentby the test suite is the ADL test suite Ver. 1.1.3, and the following function wasused:
・SCORM 2004 LMS conformance test
・SCORM 2004 content package conformance test(2) Participating product
3 LMS products and 7 content products from 4 countries were participated in
the experiment. Table 3-13 shows the participating bodies and product names.
Due to protection of enterprise secret, product names were not indicated inexperiment results, etc. but fictitious name.
Table 3-13
Country
nameParticipating body and product list

Participating body nameExperimenter

LMS productContent productProduct nameProduct nameCompaqMr. Shingo Shibata

AcademicWare

InfomationTechnolog
y-skillCheck

NTT Resonant

Mr. Yoichi Kosaka

Open Source
LMS-

Sangyo Noritsu University Mr. Hiroshi Miyauchi-

test2.zip

AEN-WG1

Toida member-

BaseballKorea

ALEX IT

Mr. Park Choonwon

NetCampus21

TestContents

Singapore

Temasek Polytechnic

Mr. Lim Kin Chew

Japan

USA

ADL

Content -C(*1)-

ADL test content-A(*1)-

ADL test content –B(*1)

3 products

7 products

Ms. Jennifer Brooks

Total(*1) Since these were not scheduled for participating in experiment, they wereparticipated in only between products.

3-１２

3.4.2Compatibility experiment by test suite

As the result of experiment that the test suite was used, failure of 1 of 5
content products was detected. The failed product ratio is 25% (2 failed
products/8 products) high. The experiment result and failure are shown in
Tables 3-14, 3-15 and 3-16 respectively.
Table 3-14

Experiment result of LMS by SCORM 2004 test suite

Product nameLMS product - I

LMS product - J

LMS product - K

Compatibility

○

x

○

None

1 (F-04)

None

Number ofproblem occurred

Table 3-15

Testing result of content by SCORM 2004 test suite

Product nameProduct - L

Product - M

Product - N

Product - P

Product - Q

Compatibility

x

○

○

○

○

1 (F-05)

None

None

None

None

Number ofproblem occurred

Table 3-16
Order No.

Failure list by SCORM 2004 test suite
Phenomena, etc.

F-04

Proper value cannot be obtained with Get
Value

F-05

SCO does not skip properly

3.4.3

Cause
SCORM standard
infringement
SCORM standard
infringement

Interoperability validation experiment between LMS and content products

In 15 combination experiment of 3 LMS products and 7 content products of 15
combinations, problem occurred in 2 combinations. The failure occurrence ratio
is 13 % (2 failures/15 combinations) high. Both 2 combinations are out of
SCORM standard, and any problem related to the SCORM standard did notoccur. Concerning the product that failure was detected with the testing of thetest suite, this experiment was performed using the product with failure
corrected. As the result, any failure related the SCORM standard did not occur,and it can be said that failure detection performance of the test suite is high.
Tables 3-17 and 18 show the experiment result and failure content.

3-１３

Table 3-17

Experiment result of interoperability between SCORM 2004 productsContent

LMS

Product-L

Product-M

Product-N

Product-P

Product-Q

Product-R

Product-S

Product-I

○

○

○

○

○

○

○

Product-J○△(F-06)○△(F-07)○−−

Product-K

○

○

○

−

−

−

−

○:Operated properly△:Operate but problem occurredx : Did not operate- : Not tested
Table 3-18

Failure list of interoperability between SCORM 2004 products

Order No.

Phenomena, etc.

F-06

Partial link error was observed in concerned content

F-07

In the interoperability experiment, apostrophe (‘) wasin the file name. Failure occurred when concernedcontent was imported

3.4.4

Cause
Problem of out of SCORM
standard. Under investigation
Problem of out of SCORM
standard. Under investigation

Preparation of SCORM 2004 test suite guide, etc.

3.4.4.1 Preparation purpose

For the SCORM 2004 test suite, various information can be obtained from the
ADL web site, however the following problems exist for actual utilization:・Description regarding compatibility inspection condition, method and others is
not sufficient. There is problem in the simple sequence inspection of content.
・Corresponding method and cause search method at trouble occurrence are notdescribed.
・The test know-how of efficient testing method is not described.
・Due to English inscription, it is difficult to understand (Content inspection withmany users especially becomes problem)
In order to resolve and avoid the above problems, the following 3 documents were
provided:・SCORM 2004 test suite guide・LMS inspection guide by SCORM 2004 test suite・Content inspection case sheet by SCORM 2004 test suite
3.4.4.2

Preparation content(1) SCORM 2004 test suite guide
These documents were provided as concrete guides for using the test suite

3-１４

regarding the procedure from install of the test suite to test execution, point ofcause investigation in the case error occurred and others. The following shows
the contents:
1 Test suite overview
1-1 How to obtain test suite
1-2 Operation environment of test suite
1-3 Install procedure of test suite
1-3-1 Install of Java software
1-3-2 Install of test suite
1-4 Other setting
2 compatibility inspection
2-1 Operational procedure of LMS compatibility inspection
2-1-1 LMS compatibility inspection
2-2 Operational procedure of content compatibility inspection
2-2-1 Content package compatible inspection
2-2-2 SCO runtime compatible testing
2-2-3 Metadata compatible testing
2-2-4 Manifest compatible testing
2-3 Inspection results
2-3-1 Icon explanation of test log
2-3-2 LMS compatible matrix
2-3-3 Content package compatible matrix
2-3-4 SCO compatible matrix
2-3-5 Metadata compatible matrix(2) LMS inspection guide by SCORM 2004 test suiteThis document was provided as the LMS inspection document, and is theguide for ADL document ”SCORM Conformance Requirements

Version 1.3”.

The document describes the test content of simple sequence function that
was added by SCORM 2004 in details. Since the originals only describes
imsmanifest setting and SCO operation, the detailed sequence (why does it
operate like so?) is unknown, and it made cause investigation easy at trouble
occurrence by describing in the detailed explanation of sequencing. This
contents are the same as the contents of ADL document.
(3) Content inspection case sheet by SCORM 2004 test suiteThis document was provided as the guide for content inspection. The
inspection cases regarding the various types of content that the sequencing
was used, were stated. By using the content type (template) described in

3-１５

“Practical guide for content developer”, the inspection method on every eachcontent type was concretely stated. The following shows the contents:
1 Preface
2 Case of single SCO (template1, template 2 and template 3)
3 Case of multiple SCO (Template 4)
4 Case of having common learning target in multiple SCO (template 5 andtemplate 6)
5 Case of having interim node activity in multiple SCO (template 7 andtemplate 8)
6 Case of that multiple distribution used common learning target was set( template 9)
7 Case of that common learning target is referred from SCO and is written(Template10)
3.4.5 Consideration of experiment result
3.4.5.1 Interoperability quality of each country productThe interoperability quality of each country product is not high. The failure
ratio (number of non-conformed products/number of products tested) of the
SCORM testing product found with the compatibility testing of test suite, is25%. The failure ratio (number of failure occurred combination/number of
combination tested) between products found with the interoperability
experiment is also 13% high, and the interoperability quality of each country
product is low.
3.4.5.2 Effectiveness and operability of test suite(1) The test suite is effective as the compatibility inspection tool.
2 infringed SCORM standard products were detected with the compatibility
testing used the test suite. While, failure related to the SCORM standard was
not detected in the interoperability experiment between products. From thisresult, it can be said that the test suite is effective as the compatibility
inspection tool.
(2) Operability, etc. of test suite(a) Time required for testing
The time required for LMS testing is at least 4 hours in the all item (59items) testing. Testing every each test item is required before entire testing,and it will therefore take many hours for testing. Furthermore, since theidentification whether such cause is product failure or freeze is difficult
when trouble occurs, it will take more time.
(b) High performance PC is required for testing
Since the log information of inspection result is approximate 20MB which ishigh, it takes a time for processing and frequent freeze occurs. It is seemedthat the cause of freeze will be time over of browser. Using the highperformance PC is desired for inspection as freeze countermeasure.
(c) Know-how of testing execution
Through this validation experiment, the following execution know-how was
found:
・Browser should shut down every each 1 testing item completion. The
number of testing items is 59.
・Every each 1 testing item should be certainly performed. Continuous

3-１６execution should not be performed.
・In order to obtain the conformance, all through testing is required.
3.5

Holding of ALIVE 2005

3.5.1

Conference overview

The International Conference Alive 2005 was held in Tokyo in December 14 and
15 , 2005. The participants were 14 from 11 overseas countries and 11 from Japan.
Ms. Brooks who is a person in charge of Co-Lab and visited from ADL
participated in the conference, and addressed comments to each country opinion.
Major implementation contents are the interoperability experiment between
products and information exchange. Tables 3-19 and 3-20 show the participants
and agenda.
Table 3-19

Participant list

Country nameCambodia

Mr. Sok Tha

Indonesia

Mr. Binsar Siagian

Body belongs to
Ministry of Education, Youth and Sport
Technical Education Development Center

China

Mr.Ronghuai HUANG

Beijing Normal University

Mr.Ju Hyung Lee

Dunet Inc.

Mr.ChoonWon Park

AlexIT Cp.,Ltd

Ms.Khampheng Phathadavong

Sengsavanh College

Mr. David AsirvathamMultimedia University

Mr. Muhammad Hasan

Multimedia University

Prof. Rufino MananghayaPhilippine e-learning Society

Dr. Benito Teehankee

Philippine e-Learning Society

Singapore

Mr. Lim Kin Chew

Learning Standards Technical Committee

ThailandDr.Niracharapha hongdhamachart

MICT

Viet Nam

Mr.Tin Nguyen Ba
Ms Jennifer Brooks
Takeshi Kumazawa

New Century Soft Company
Alexandria ADL

Kiyoshi Nakabayashi

NTT Resonant

Yoichi Kosaka
Hiroshi Miyauchi
Shingo Shibata

NTT Resonant
Sangyo Notitsu University
CompaqTooru Nakajima- ditto -Jun hua Ruan

- ditto -Ken-ichi Tsuji, Toshiyuki Kobayashi,
Yosuke Yoshimura, Toshio Munemoto

e-Leaning Consortium, Japan

Korea
Laos
Malaysia
Philippine

USA

Japan

Secretariat

Name

Co-Laboratory

Human Science

3-１７

Table 3-20

AgendaDate and time

Dec. 14

14:00-14:30

Explanation of schedule, experiment procedure, etc.

14:30-15:00

Recovery and distribution of experiment content

15:00-18:00

Experiment implementation

9:00-9:30

Addressing preparation

9:30-11:00

Addressing of experiment result

11:00-11:20

Summery of experiment result

11:20-12:00

Opinion exchangeDec. 15

3.5.2<Implementation content＞

Information exchange content
Table 3-21
Opinion exchange content

Speaker（country name）
Ms Brooks (USA)

Mr.
Lim(Singapore)
Ms Brooks

Kim

Mr. Hasan (Malaysia)
Ms Brooks (USA)Chew

・ SCORM 2004 is also propagating in the United State. The
Department of Defense recommends to introduce SCORM
2004.
・The number validation of LMS of SCORM 2004 is 14 products.
・ SCORM 2004 test suite has not been changed except
for simple sequence.
・ I’m pleased to have heard yesterday and today presentationand to have observed the situation that the testing wascomparatively successful.
・Various problems existed on LMS, however this is very complex
and is not easy.
・There are many problems, however I would like to evaluate
aggressive activity by each country.
・Since SCORM 2004 is extremely complex, an important thing isthat each country should emphasize to tackle with it.
・When is SCORM 2004 completed? 2 years or 5 years later?
・Currently, I cannot say in details since content is not fixed,
however it is an important to aim at completion by continuous
effort in future.
・Corresponding to needs between communities, making an effort
and achieving are important.
・What do you think of Sequencing?
・We need to debate terminology issue in future.
・Since Sequencing has relation with LMS, it is necessary toconsider such point.

3-１８

Mr.
Lim
(Singapore)

Kim

Chew

Ms Brooks (USA)Mr. Hasan (Malaysia)

Ms Brooks (USA)Dr. Benito (Indonesia)
Ms Brooks (USA)Dr. Benito (Indonesia)

Prof. Ruffin (Philippine)
Ms Brooks (USA)Dr. Benito (Indonesai)
Mr. Lim Kim Chew (Singapore)
Dr. Benito (Indonesia)

Mr. Park (Korea)
Ms Brooks (USA)Dr. Benito (Indonesia)
Mr. Tsuji (Japan)

Ms Brooks (USA)・ Instruction design is creating various problems. We canunderstand the value, but…..
・ Advanced countries have strong will for progress, however
developing countries do not unfortunately have such behavior,and does not especially care about progress.
・ SCORM 2004 needs further authoring tool regarding
collaboration of instruction design and program.
・Many points of instruction tool are not known. Due to this, isthere much easier tool for designer?
・ I also agree with it. I think such manual will be needed
immediately.
・I think it is true that was checked with SCORM. Especially in thedeveloping countries…….
・However, I think SSP is very effective.
・For element, I think further challenge is needed.
・Similar thing can be said for LMS.
・SCORM tool of ADL is extremely expensive.
・The US government realizes such issue, and is making an effort
for cost down.
・When looking at the ADL website, there are so many agreement
items, and it is therefore extremely restricted.
・Why do you choose SCORM? I do not understand the reason.
・What is the strategy? Is it business or education?
・ Since we have not challenged in earnest, we have no idea ofthe meaningfulness. Are tool, etc. needed?
・ I think tool is needed. Tool is helpful, and it is useful for
e-Learning.
・I think we should choose SCORM for Asian countries.
・I do not think other countries purchase special SCORM.
・Any countries will not choose complicated SCORM. Especially,certificate certifying the compatibility will be needed.
・If you have certificate, you can sell it as product, however if not,you cannot sell it. This is my past experience.
・Speaking of Korean presentation, I understand such SCORM
development needs US$6 million, but who pays such large
amount?
・Since e-Learning can be useful for business, government will
support it.
・If standard is used, it will contribute to normal cost down.
・Anyway, we have many experiences of SCORM
・I think SCORM has surely effectiveness.
・Client has not inquired about SCORM 2004.
・What are you thinking of future ADL challenge?
・Why has not SCORM been propagated?
・Presently, we are reviewing SCORM 2004, and we are understatus of trial and error.
・In addition, since 2004 will be reviewed every 2 - 3 years,change of technology is tough work.

3-１９

3.6

Summery of activity result

3.6.1

Activity result(1) Interoperability quality of each country product
The

SCORM

testing

product

non-conformance

ratio(numberof

non-conformed products/number of product tested) of the SCORM 1.2
products of each AEN country and the SCORM 2004 compatible products are
17% and 25% respectively, and the compatibility to the SCORM standard islow. It is mandatory that the compatibility testing shall be aggressively
performed with the test suite to increase the compatibility.
As the result of interoperability experiment between products, 4 troubles (2troubles of SCORM 1.2 product and 2 troubles of SCORM 2004 products)
arisen from problem of out of SCORM standard were detected. In order toimprove the interoperability quality, it is also necessary to correspond withproblems of out of SCORM standard. eLC has already stated and announced
the know-how regarding the interoperability trouble found with questionnaire
surveillance, etc. in the “Application technology of SCORM interoperability
improvement seen in case study” document, regardless of within or out of the
SCORM standard. According to the questionnaire surveillance, the number ofinteroperability problem occurrence has been decreasing every year, and it isseemed that effectiveness has been proved.
(2) Interoperability quality of authoring tool productFor 5 commercial authoring tool products that declare SCORM 1.2
compliance, the SCORM compatibility testing was performed, and it was
found that 1 product was SCORM standard infringement. For the authoring
tool, it is also necessary to perform the compatibility inspection by the test
suite. Furthermore, if further improvement of the interoperability is aimed at,
Establishment of the validation system for authoring tool is desired.
(3) Effectiveness and operability of test suite
It was found that the test suite is extremely effective as inspection tool. 3
products of compatibility infringed products were found with this validation
experiment. Failure related to the SCORM standard of the products without
compatibility infringement in the interoperability experiment between products
did not occur even in the interoperability experiment between products.
It was found that the test suite for SCORM 2004 has problem on operability,
etc. As the countermeasure, the “SCORM 2004 test suite guide”, etc. were
provided.
(4) Preparation of “SCORM 2004 test suite guide”, etc.

3-２０Documents being supplied by ADL are insufficient description for test
condition, method of the test suite and countermeasure at failure occurrence,
resulting in inspection hindrance. As the countermeasure, 3 types of
document such as “SCORM 2004 test suite guide”, “LMS inspection guide by
SCORM 2004 test suite” and “Content inspection case guide by SCORM 2004
test suite” were provided. In order to effectively use the test suite, this book is
considered to be mandatory.
(5) Information exchange with each country and ADL
It was found that each AEN country is aggressively introducing SCORM.
While, it was also found that there are argument opinion regarding
effectiveness and its scope of the SCORM standard. Ms Brooks of ADL
advised us of each country opinion.
3.6.2

Future issue(1) SCORM compatibility improvement of product by utilization promotion of test
suite
As the result of this validation experiment, it might be possibility that the
SCORM compatible product that fringes to the SCORM standard is available
in the market. It is therefore necessary to promote interoperability
improvement activity for utilization, etc. of the test suite.
(2) Continuous implementation of ALIVE
Many participants desire the interoperability experiment between eachcountry products and continuous implementation of information exchange
regarding SCORM standard and the interoperability.

3-２１