MEMORANDUM ON PERMISSION TO USEJURISIN COLIEE 2020 TASK PARTICIPANT DATA COLLECTION(E)(FOR COLIEE 2020 TASK PARTICIPANTS)

This Memorandum is made and entered into this month [] day [], year [] between the JURISIN COLIEE Organizing Group(hereinafter “COG”) and _____________________________________________________________________________________________________________________________________________________ (hereinafter “USER/USER’SRESEARCH GROUP”) with regard to the JURISIN COLIEE Legal Information Entailment Competition task: Bar exam English Data Collection(E)which COG provides at the JURISIN COLIEE 2020 workshop.

Article 1. Description of Data
1.
2.
3.
4.

Task Data(E) is defined as data which appears in Section 1 of the Appendix attached herein.
Raw Submission Data is defined as data that the results which were submitted by the task participants of the JURISIN COLIEE 2020
workshop.
Evaluation Data is data obtained by using the basic evaluation tools defined by COG against “Raw Submission Data”.
Data Collection(E) consists of the Task Data(E), Raw Submission Data, Evaluation Data, Outputs of Natural Language Processing Tools, andPROLEG logic examples.

Article 2. Retention of Rights
1.
2.
3.

The copyright holders of the Task Data(E) are defined in Section 2 of the Appendix attached herein.
USER/USER’S RESEARCH GROUP retains all intellectual property rights relating to the technology and systems developed by
USER/USER’s RESEARCH GROUP using the Data Collection(E).
COG retains all intellectual property rights relating to analytical results obtained by COG and improvements on the Data Collection(E) andsuch made by COG based on data submitted by USER/USER’s RESEARCH GROUP.

Article 3. Permission for Use
COG hereby permits USER/USER’S RESEARCH GROUP to use the Data Collection(E).

Article 4. Scope of Authorized Use
1.

2.

3.
4.
5.

USER/USER’S RESEARCH GROUP shall use the Data Collection(E) solely for the purpose of accomplishing task set out in the JURISIN
COLIEE 2020 Workshop and for the purpose of research related to the task until and inclusive of the last day of November 2020, the end of
the term of JURISIN COLIEE 2020 Workshop.
USER/USER’S RESEARCH GROUP shall restrict that uses of the Data Collection(E) are done only by the following Research Representative(hereinafter referred to as “Representative”) and by those who belong to the organization (regardless of its name, such as “laboratory”, “group”,or “project”) that Representative is a member of, and do research jointly with Representative or are graduate students and the like
Representative supervises (hereinafter referred to as “Group”).
Research Representative:
Name:
Title:
Affiliation:
USER/USER’S RESEARCH GROUP shall not sell, lend, publish, distribute, or make transmittable to a third party the Data Collection(E),copies of the Data Collection(E), or processed data which would enable a third party to reproduce the Data Collection(E).
USER/USER’S RESEARCH GROUP shall use the Data Collection(E) solely for the purpose of research related to legal information retrieval
and natural language processing after the end of the term of JURISIN COLIEE 2020 workshop.
USER/USER’S RESEARCH GROUP shall maintain a list of users and make this list available to COG without delay upon request from COG.

Article 5. Method of Provision
COG will provide the Data Collection(E) to USER/USER’S RESEARCH GROUP using a method which is defined in Section 3 of the Appendix
attached herein.

Article 6. Presentation of Knowledge
1.
2.

3.

4.

5.

6.
7.

USER/USER’S RESEARCH GROUP may present research findings concerning knowledge obtained using the Data Collection(E) provided
that the aforementioned presentation is within the limits of this Memorandum.
USER/USER’S RESEARCH GROUP may quote in part data included in the Data Collection(E) if and only if the aforementioned data isrequired to describe USER/USER’S RESEARCH GROUP’s own research. In such cases, USER/USER’S RESEARCH GROUP shall notviolate the rights of the authors, publishers and others protected by copyright.
When USER/USER’S RESEARCH GROUP shall present research findings concerning knowledge obtained using the Raw Submission Dataand Evaluation Data, USER/USER’S RESEARCH GROUP shall clearly mention USER/USER’S RESEARCH GROUP’s use of the data andclearly state the experimental nature of the data and the data collection procedure of the data.
USER/USER’S RESEARCH GROUP shall always reference the JURISIN COLIEE 2020 workshop proceedings and clearly state in the paper,
thesis or other presentation that USER/USER’S RESEARCH GROUP presents (a) USER/USER’S RESEARCH GROUP’s use of the Bar
Exam Data, and (b) USER/USER’S RESEARCH GROUP’s use of the Data Collection(E).
USER/USER’S RESEARCH GROUP shall submit to COG: (a) a publication report including bibliographic information of the publication (thetitles of the publication, volume, pages, publishers’ names and dates), and (b) one (1) offprint or one (1) copy of the publication each time
when USER/USER’S RESEARCH GROUP publishes the research results using the Data Collection(E) or part of the Data Collection(E).
Before making public any information data developed using the Data Collection(E), USER/USER’S RESEARCH GROUP should receive
consent in writing from COG in advance.
USER/USER’S RESEARCH GROUP shall not use evaluation results obtained through use of the Data Collection(E) for profitable purposes
including advertising, and/or defamatory or slanderous purposes.

Article 7. Duration of Memorandum
1.

2.

3.

This Memorandum shall be in effect from the date first above written until and inclusive of the last day of November 2020. The effect of theMemorandum shall be extended automatically to another fiscal year, unless a written objection is issued either by COG or by USER/USER’SRESEARCH GROUP more than one month before the expiration date of the Memorandum. This shall apply to consecutive fiscal years as well.
Notwithstanding the preceding paragraph, if permission for use of the Raw Submission Data and Evaluation Data is requested byUSER/USER’S RESEARCH GROUP which shall extend the effect of the Memorandum, USER/USER’S RESEARCH GROUP shallseparately enter into a new Memorandum with COG, in addition to the written notice more than one month before the expiration date of theMemorandum.
If USER/USER’S RESEARCH GROUP would not like to automatically renew the original terms and the renewal terms, after expiration of theterm, USER/USER’S RESEARCH GROUP shall promptly delete the Data Collection(E) from any computer or media onto which it has been
copied. If the organizational structure of USER/USER’S RESEARCH GROUP or the affiliation of USER/USER’S RESEARCH GROUP ischanged, USER/USER’S RESEARCH GROUP shall report the change to COG without delay and, if necessary, shall sign anotherMemorandum.

Article 8. Submission of Reports
1.
2.

USER/USER’S RESEARCH GROUP shall submit to COG a report on research outcomes in accordance with procedures specified at theJURISIN COLIEE 2020 Workshop.
USER/USER’S RESEARCH GROUP shall submit to COG a report concerning research activities conducted by USER/USER’S RESEARCHGROUP during the duration of the Memorandum using the Data Collection(E) more than one month before the expiration date of theMemorandum.

Article 9. Termination of Data Use
1.

2.

If the use of data is deemed to be in violation of the Memorandum, USER/USER’S RESEARCH GROUP shall, upon instructions from COG,
immediately discontinue use of the Data Collection(E) and promptly delete the Data Collection(E) and all data obtained by processing the DataCollection(E) from any computer or media onto which it has been copied. USER/USER’S RESEARCH GROUP shall submit a documentattesting to deletion of the data to COG.
If there is a request to discontinue the use of data from the copyright holder(s) or their executor(s) of the Data Collection(E), USER/USER’SRESEARCH GROUP must delete said data from any computer or media onto which it has been copied upon instructions from COG.
USER/USER’S RESEARCH GROUP shall submit a document attesting to deletion of the data to COG.

Article 10. Exemption from Liability
COG and the copyright holder(s) or their executor(s) of the Data Collection(E) shall not for any reason whatsoever be responsible for losses arisingfrom USER/USER’S RESEARCH GROUP’s use of the Data Collection(E).

Article 11. Undefined Issues
Should any issues undefined in this Memorandum arise, both parties shall confer in good faith and resolve such issues.

Article 12. Jurisdiction
All disputes relevant to this Memorandum shall be subject to the exclusive jurisdiction of the Tokyo District Court of Japan as court of first instance.
The validity, construction and performance of this Memorandum shall be governed by, and interpreted in accordance with, the law of Japan.
Both parties hereto confirm to this Agreement to be executed in duplicate by their duly authorized officers as of the date first above written.
Signed for and on behalf of

COG:

JURISIN COLIEE Competition Organizer

Signature:
Name (in print):
Title:
Division:
Notice address:____________________________________________________Ken Satoh
Professor
National Institute of Informatics
2-1-2 Hitosubashi, Chiyoda-ku Tokyo, Japan 101-8430

Signed for and on behalf ofUSER/USER’S RESEARCH GROUP:______________________________________________

Signature:______________________________________________Name (in print):______________________________________________

Title:______________________________________________

Division:______________________________________________

Notice address:____________________________________________________________________________________________

APPENDIX-JURISIN COLIEE 2020 Legal Information Entailment: Bar Exam English DataTASK PARTICIPANT DATA COLLECTION(E)(FOR JURISIN COLIEE WORKSHOP 2020 COMPETITION PARTICIPANTS)

Section 1.
The competition formal name is JURISIN COLIEE Workshop 2020 Competition on Legal Information Extraction/Entailment: Bar Exam English
Data, and the term of “Task Data(E)” consists of JURISIN COLIEE 2020 Workshop Training/Test Data written in English.
2. JURISIN COLIEE 2020 Workshop Training/Test Data written in English consists of English translation of a set of queries, relevant civil lawarticles, and Yes/No answer of the queries that COG created using Bar Exam Data of Japan which appears in Table A, English translation of Japan
Civil Law Articles provided by Japanese Law Translation Database System of Ministry of Justice in Japan, and annotations that COG assigned tothe data. The data will be available dependent on the participating task.
Table A
A.1 Bar Exam Data
A.1.1 The training data in the following list
Bar Exam(Short Answer Test) Civil Code part published in 2006
Bar Exam(Short Answer Test) Civil Code part published in 2007
Bar Exam(Short Answer Test) Civil Code part published in 2008
Bar Exam(Short Answer Test) Civil Code part published in 2009Bar Exam(Short Answer Test) Civil Code part published in 2010
Bar Exam(Short Answer Test) Civil Code part published in 2011Bar Exam(Short Answer Test) Civil Code part published in 2012Bar Exam(Short Answer Test) Civil Code part published in 2013

Bar Exam(Short Answer Test) Civil Code part published in 2014

Bar Exam(Short Answer Test) Civil Code part published in 2015

Bar Exam(Short Answer Test) Civil Code part published in 2016
Bar Exam(Short Answer Test) Civil Code part published in 2017Bar Exam(Short Answer Test) Civil Code part published in 2018

A.1.2 The test data for Task 1 and Task 2 in the following list
Bar Exam(Short Answer Test) Civil Code part published in 2019

Section 2.
COG retains derivative copyright of translations of Bar Exam Task Data. COG retains all rights under the Copyright Law relating to annotations.

Section 3.
COG will provide the Data Collection(E) to USER/USER’S RESEARCH GROUP by transferring the data files electronically.