MEMORANDUM ON PERMISSION TO USE
NTCIR-13 QA Lab for Entrance Exam: Essay Question Subtask TASK PARTICIPANT TEST COLLECTION(FOR NTCIR PARTICIPANTS)

This Memorandum is made and entered into this [] day [], year [] between Inter-University Research Institute Corporation
Research Organization of Information and Systems, a corporation incorporated in Japan and having its registered office at 4-3-13 Toranomon,
Minato-ku,
Tokyo,
Japan
105-0001(hereinafter“ROIS”)
and___________________________________________________________________________________________________________________ (hereinafter “USER/USER’S RESEARCHGROUP”) with regard to the NTCIR-13 QA Lab for Entrance Exam: Essay Question Subtask Task Participant Test Collection which National
Institute of Informatics, a division of ROIS provides and organized at the NTCIR 13 .

Article 1. Description of Data
1.
2.
3.
4.
5.

Document Data is defined as data which appears in Section 1 of the Appendix attached herein.
Task Data consists of Topic Data and Answer Data and is defined as data which appears in Section 2 of the Appendix attached herein.
Raw Submission Data is defined as data that the results which were submitted by NTCIR-13 QA Lab for Entrance Exam: Essay Question Subtask task participants
and NTCIR-12 QA Lab for Entrance Exam: Essay Question Subtask task participants of the NTCIR that ROIS organized.
Evaluation Data is data obtained by using the basic evaluation tools defined by ROIS against “Raw Submission Data”.
Test Collection consists of Document Data, Task Data, Raw Submission Data and Evaluation Data.

Article 2. Retention of Rights
1.
2.
3.
4.

The copyright holders of Document Data are defined in Section 3 of the Appendix attached herein.
The copyright holders of Task Data are defined in Section 4 of the Appendix attached herein.
USER/USER’S RESEARCH GROUP retains all intellectual property rights relating to the technology and systems developed by USER/USER’S RESEARCHGROUP using the Test Collection.
ROIS retains all intellectual property rights relating to analytical results obtained by ROIS and improvements on the Test Collection and such made by ROIS basedon data submitted by USER/USER’S RESEARCH GROUP.

Article 3. Permission for Use
ROIS hereby permits USER/USER’S RESEARCH GROUP to use the Test Collection.

Article 4. Scope of Authorized Use
1.
2.

3.
4.

USER/USER’S RESEARCH GROUP shall use Test Collection solely for the purpose of accomplishing tasks set out in the NTCIR and for the purpose of research
related to the tasks.
USER/USER’S RESEARCH GROUP shall restrict uses of DATA are done only by the following Research Representative (hereinafter referred to as“Representative”) and by those who belong to the organization (regardless of its name, such as “laboratory”, “group” or “project”) that Representative is a memberof, and do research jointly with Representative or are graduate students and the like Representative supervises (hereinafter referred to as “Group”).
Research Representative:
Name:
Title:
Affiliation:
USER/USER’S RESEARCH GROUP shall not sell, lend, publish, distribute, or make transmittable to a third party the Test Collection, copies of the Test Collection,or processed data which would enable a third party to reproduce the Test Collection.
USER/USER’S RESEARCH GROUP shall maintain a list of users and make this list available to ROIS without delay upon request from ROIS.

Article 5. Method of Provision
ROIS will provide the Test Collection to USER/USER’S RESEARCH GROUP using a technologically appropriate method.

Article 6. Presentation of Knowledge
1.
2.
3.

4.

5.
6.
7.

USER/USER’S RESEARCH GROUP may present research findings concerning knowledge obtained using the Test Collection provided that the aforementioned
presentation is within the limits of this Memorandum.
USER/USER’S RESEARCH GROUP may quote in part data included in the Test Collection if and only if the aforementioned data is required to describe
USER/USER’S RESEARCH GROUP’s own research. In such cases, USER/USER’S RESEARCH GROUP shall not violate the rights of the authors, publishers
and others protected by copyright.
When USER/USER’S RESEARCH GROUP shall present research findings concerning knowledge obtained using the Raw Submission Data and Evaluation
Data, USER/USER’S RESEARCH GROUP shall clearly mention USER/USER’S RESEARCH GROUP’s use of the data and clearly state the experimental
nature of the data and the data collection procedure of the data.
USER/USER’S RESEARCH GROUP shall always reference the NTCIR proceedings and clearly state in the paper, thesis or other presentation that
USER/USER’S RESEARCH GROUP presents: USER/USER’S RESEARCH GROUP’s use of the Test Collection, the Document Data, the Topic Data and theAnswer Data. USER/USER’S RESEARCH GROUP shall present their use of the Document Data with procedures specified by the copyright holders ofDocument Data and ROIS.
USER/USER’S RESEARCH GROUP shall submit to ROIS: (a) a publication report including bibliographic information of the publication (the titles of thepublication, volume, pages, publishers’ names and dates), and (b) one (1) offprint or one (1) copy of the publication each time when USER/USER’S RESEARCHGROUP publishes the research results using the Test Collection or part of the Test Collection.
Before making public any information data developed using the Test Collection, USER/USER’S RESEARCH GROUP should receive consent in writing fromROIS in advance.
USER/USER’S RESEARCH GROUP shall not use evaluation results obtained through use of the Test Collection for profitable purposes including advertising,and/or defamatory or slanderous purposes.

Article 7. Duration of Memorandum
1.
2.

This Memorandum shall be in effect from the date first above written until and inclusive of the last day of September 2018.
After expiration of the term as defined in the foregoing 1 hereof, USER/USER’S RESEARCH GROUP shall promptly delete the Document Data and Topic
Data from any computer or media onto which it has been copied.

2016-11-09

3.

4.

With regard to Answer Data, Raw Submission Data and Evaluation Data, the effect of the Memorandum shall be extended automatically to another fiscal year,
unless a written objection is issued either by ROIS or by USER/USER’S RESEARCH GROUP more than one month before the expiration date of theMemorandum. This shall apply to consecutive fiscal years as well. If USER/USER’S RESEARCH GROUP would not like to automatically renew the originalterms and the renewal terms, after expiration of the term, USER/USER’S RESEARCH GROUP shall promptly delete the Test Collection from any computer ormedia onto which it has been copied.
If the organizational structure of USER/USER’S RESEARCH GROUP or the affiliation of USER/USER’S RESEARCH GROUP is changed, USER/USER’SRESEARCH GROUP shall report the change to ROIS without delay and, if necessary, shall sign another Memorandum.

Article 8. Submission of Reports
1.
2.

USER/USER’S RESEARCH GROUP shall submit to ROIS a report on research outcomes in accordance with procedures specified at the NTCIR.
USER/USER’S RESEARCH GROUP shall submit to ROIS a report concerning research activities conducted by USER/USER’S RESEARCH GROUP during
the duration of the Memorandum using the Test Collection more than one month before the expiration date of the Memorandum.

Article 9. Termination of Data Use
1.

2.

If the use of data is deemed to be in violation of the Memorandum, USER/USER’S RESEARCH GROUP shall, upon instructions from ROIS, immediately
discontinue use of the Test Collection and promptly delete the Test Collection and all data obtained by processing the Test Collection from any computer or media
onto which it has been copied. USER/USER’S RESEARCH GROUP shall submit a document attesting to deletion of the data to ROIS.
If there is a request to discontinue the use of data from the copyright holder(s) or their executor(s) of the Test Collection, USER/USER’S RESEARCH GROUP
must delete said data from any computer or media onto which it has been copied upon instructions from ROIS. USER/USER’S RESEARCH GROUP shallsubmit a document attesting to deletion of the data to ROIS.

Article 10. Exemption from Liability
ROIS and the copyright holder(s) or their executor(s) of the Test Collection shall not for any reason whatsoever be responsible for losses arising from USER/USER’SRESEARCH GROUP’s use of the Test Collection.

Article 11. Undefined Issues
Should any issues undefined in this Memorandum arise, both parties shall confer in good faith and resolve such issues.

Article 12. Jurisdiction
All disputes relevant to this Memorandum shall be subject to the exclusive jurisdiction of the Tokyo District Court of Japan as court of first instance. The validity,construction and performance of this Memorandum shall be governed by, and interpreted in accordance with, the law of Japan.

Both parties hereto have caused this Agreement in duplicate to be executed by their duly authorized officers as of the date first above written.
Signed for and on behalf ofROIS:Inter-University Research Institute Corporation
Research Organization of Information and Systems

Signature:
Name (in print):
Title:
Division:
Notice address:___________________________________________________
Masaru Kitsuregawa
Director General
The National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda-ku, TokyoJAPAN101-8430

Signed for and on behalf ofUSER/USER’SRESEARCHGROUP:
Signature:___________________________________________________Name (in print):
Title:
Division:
Notice address:_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

2016-11-09

APPENDIX- NTCIR-13 QA Lab for Entrance Exam: Essay Question Subtask TASK PARTICIPANT TEST COLLECTION(FOR NTCIR 13 PARTICIPANTS)
Section 1.
A formal name is called NTCIR-13 QA Lab for Entrance Exam Task Textbook Data with, and the term of “Document Data”, is defined as data
selected and composed by USER/USER'S RESEARCH GROUP from data which appears in Table A that ROIS is granted permission by therightful holder(s) thereof, whose names appear in Section 3, to permit use of the data for the participants as a collaborator of QA Lab for Entrance
Exam: Essay Question Subtask (Japanese) in the NTCIR Workshop 13.
Table A (please tick the resources you are requesting)□ A.1 Japanese Textbook Corpus1 – Tokyo Shoseki World History Subset :A.1.1 the data below that Tokyo Shoseki Co., Ltd provided for ROIS and appears in the following list
・World History A (『世界史 A』) (published in Fiscal Year 2008) (世A・020)
・World History B (『世界史 B』) (published in Fiscal Year 2007) (世B・013)
・New World History B (『新選世界史 B』) (published in Fiscal Year 2007) (世B・012)
A.1.2 Annotations that ROIS assigned to the data that appears in List A.1.1.
A.1.3 Indexed Data that ROIS assigned to the data that appears in List A.1.1.
□ A.2 Japanese Textbook Corpus2 - Yamakawa Shuppansha World History Subset :A.2.1 the data below that Yamakawa Shuppansha Ltd., provided for ROIS and appears in the following list
・Detailed explanation World History (『詳説世界史』(世 B 016) ): Text Data of the text and the explanatory note

A.2.2 Annotations that ROIS assigned to the data that appears in List A.2.1.
A.2.3 Indexed Data that ROIS assigned to the data that appears in List A.2.1.
Section 2.
1.
A formal name is called NTCIR-13 QA Lab for Entrance Exam: Essay Question Subtask Task Data with, and the term of “Task Data”,
consists of “Topic Data” and “Answer Data” and is defined as data selected and composed by USER/USER'S RESEARCH GROUP fromdata which appears in Table B.
Table B (please tick the resources you are perticipating)□ B.1 NTCIR-13 QA Lab for Entrance Exam: Essay Question Subtask (Japanese) Task Data
□ B.2 NTCIR-13 QA Lab for Entrance Exam: Essay Question Subtask (English) Task Data
2.

“Topic Data” consists of Todai-Past Exam (Essay Question) Topic Data and Answer Data 1 and Todai-Past Exam (Essay Question) Topic
Data and Answer Data 2 and is defined as data selected, translated and composed by ROIS from data which appears in Table C that ROIS is
granted permission by the rightful holder(s) thereof to permit use of the data for the purpose of accomplishing tasks set out in NTCIR 13 and forthe purpose of research related to the tasks to the participants.
Table C
C.1 Todai-Past Exam (Essay Question) Topic Data and Answer Data 1:
C.1.1 the Digital data (in pdf format and in word format), the text data and the image data of the inset of the plane of paper ofthe secondary entrance exam questions of the University that JC Educational Institute, Inc. provided for ROIS.
C.1.2 Annotations that ROIS assigned to the data that appears in List C.1.1.
C.2 Todai-Past Exam (Essay Question) Topic Data and Answer Data 2:
C.2.1 the Digital data (in pdf format and in word format), the text data and the image data of the inset of the plane of paper ofthe secondary entrance exam questions of the University that was executed after 2012.
C.2.2 Annotations that ROIS assigned to the data that appears in List C.2.1.

3.

“Answer Data” consists of a set of answer data and answer nugget data that ROIS created using Todai-Past Exam (Essay Question) Topic Dataand Answer Data 1 and Todai-Past Exam (Essay Question) Topic Data and Answer Data 2.

Section 3.
1. Tokyo Shoseki Co., Ltd retains all rights under the Copyright Law relating to Japanese Textbook Corpus1 – Tokyo Shoseki World History
Subset, including in Document Data which appears in Table A.1, except quotations of which copyright holder(s) is defined clearly in the said
data as a third party and the annotations. ROIS retains all rights under the Copyright Law relating to a set of the annotations.
2. Yamakawa Shuppansha Ltd., retains all rights under the Copyright Law relating to Japanese Textbook Corpus2 - Yamakawa Shuppansha
World History Subset, including in Document Data which appears in Table A.2, except part of which copyright holder(s) is a third party and theannotations. ROIS retains all rights under the Copyright Law relating to a set of the annotations.
Section 4.
The Copyright Law relating to Todai-Past Exam (Essay Question) Topic Data and Answer Data 1, included in Topic Data which apprears in1.
Table C.1: (i) JC Educational Institute, Inc. retains all rights under the Copyright Law relating to the data, except relating to the examinationquestions, each work that the questions contain and the annotations. (ii) ROIS retains all rights under the Copyright Law relating to theannotations.
2.
ROIS retains all rights under the Copyright Law relating to Todai-Past Exam (Essay Question) Topic Data and Answer Data 2, included inTopic Data which appears in Table C.2, except relating to the examination questions, each work that the questions contain.
3.
ROIS retains all rights under the Copyright Law relating to the Answer Data.
4.
However, Task Data related to Todai-Past Exam (Essay Question) Topic Data and Answer Data 1 and Todai-Past Exam (Essay Question)
Topic Data and Answer Data 2, and all rightful holders of Todai-Past Exam (Essay Question) Topic Data and Answer Data 1 andTodai-Past Exam (Essay Question) Topic Data and Answer Data 2 under the Copyright Law are not responsible for their content.
2016-11-09