Sketch Engine

SRDANOVIĆ ERJAVEC Irena,

Sketch Engine

Sketch Engine
Web
1“Word Sketch”“Thesaurus”
JpWaC“Sketch Difference”
4Sketch Engine

Web
Sketch Engine

1.
1980
1080

Kilgarriff & Rundell 2002
500

1,000

20,0002000
Heid et al. 2000, Kilgarriff &Tugwell 2001
Srdanović et al.

Sketch Engine

Kilgarriff et al. 2004

2008

Sketch Engine
Web

Word Sketch

Thesaurus

Sketch Difference1Sketch Engine

2. Sketch Engine
Sketch Engine

Kilgarriff et al. 2004
Erjavec et al.

442007

WebWebSketch Engine
Sketch Engine

2.1. Sketch Engine
Web

Sketch Engine

4JpWaCWeb

Ueyama & Baroni (2005)
Bernardini, eds. 2006

(http://www.sketchengine.com)
1Web

BootCatSharoff (2006)

5

WAC Baroni &Baroni et al. 2006HTML

boilerplate removal

Web

ChaSenlemmatoken
tag Erjavec etal. 2006
.jpet al. 2007.com

Erjavec

Srdanović et al. 2008

Sketch Engine
23

URL

Web
JpWaC
2007

2

1

Sketch Engine

32

Sketch Engine

Sketch Engine

2.2.

Word Sketches

22

Word Sketch, Thesaurus

Sketch Difference
Gahl 1998

Chasencorpus query syntax ()Word Sketch

3444221

salience1

modifies_N(42)
dual

*DUAL
=modifier_Ana/modifies_N
2:"N.Ana""Aux" "Pref.*"?
2:
modifier_Ana

1:[tag="N.*"& tag!="N.Suff.*" & tag!="N.bnd.*"]1:
modifies_N

N.Ana

2:"N.Ana""Aux" "Pref.*"?
2:

Pref.*

1: [tag="N.*" & tag! ="N.Suff.*" & tag!

N.*="N.bnd.*"]N.Suff.*-Aux

N.bnd.*

--

4--

0

*

N.*
0N.g

N.Prop1

Sketch Engine
Concordance

CQL

Corpus Query

Language[word=”•”| word=””]ChaSen[word=””] [word=””] [lemma=””]3.2[tag=”N.*”]&[ word =“•”]

Word Sketch
Sketch Engine

ChaSenIPADIC)IPADIC

Sketch EngineWeb
http://tell.fll.purdue.edu/chakoshipub/index2.html

ChaSen5

ChaSenChaSenSketchEnginetoken

kanalemma

POS t ag ()POS tag -eng (-Adv.P-

N.Ana
Aux

-N.g
Aux
Aux

-Sym.p

ChaSenChaSenIPADIC

ChaSenChaSen5)

Word Sketch

ChaSenWord Sketch
Word Sketch

Concordance100

Word Sketch

ChaSenWeb

2.3. Thesaurus

Sketch Difference

Thesaurus Sketch Difference

shared triples

3tripleSrdanović et al.

2008

Thesaurus6Sketch Difference

7816,309Web

66,486

2.5

Thesaurus

2.4.

7

Sketch Difference

only pattern

8Sketch Difference

only pattern

Web
WebWeb7

Web
Web

Keller & Lapata 2003

Web

Web

JpWaC

Web
Web

Sharoff 2006 Ueyama & Baroni 2005

Web

Web
Web
Sharoff 2006

Ueyama &Baroni 2005
Web
narrative styleWebinteractive style

Web
Web
Web
Ghani et al. 2001

Web
Web
Web

Web

Web
Crystal 2006
Web
•

Web

•

Web
8

Web

3. Sketch Engine
Sketch Engine

3.1.

Sketch Engine

80

Cobuild90

Church & Hanks 1989

2000

(MI)

Word Sketch

Sketch Engine

BNC British National Corpus
Rundell, ed. 2002

Kilgarriff &Rundell (2002)
Word Sketch

Word Sketch

Word Sketch
Sketch Engine
WordSketch

Sketch Engine9Kilgarriff & Rundell

2002‘challenge’2004
Sketch Engine
3.1.1
9Word Sketch

99

modifier_Ana

Word Sketch

modifier_Ai
verb

verb

verb

verb

9

‘initiation’‘trial’-10‘challenge to something/somebody‘Word Sketch

Concordance

10

CQL

[word="{0,3}Concordance"] []{0,3} [word="0113"]

token(3199•••1011

Word Sketch

jaSlo200611

Erjavec et al.

3.1.2
2004

2004

Word Sketch
10

Word Sketch1)

2)3)

4)

1)
1,180364

Sketch Engine
222

Sketch EngineSketch
EngineSketch Engine122)

Word Sketch

Word Sketch
EngineSketchWebSketch Engine

3)

Word Sketch

Word Sketch

131212

Word Sketch

4)
Word Sketch
EngineThesaurus

Sketch DifferenceAB

SketchB
A
Sketch Difference

14A

WebWeb

Word Sketch

Sketch Engine

3.2.

Sketch Engine

Sketch Engine
Word Sketch Thesaurus Sketch Difference

Concordance•suffix (•suffix_base•bound_V•

V_bound)prefixprefix_base

suffixbound_V

V_bound

Sketch Difference/15/WordSketch

Word Sketchlemma2)

ConcordanceConcordance

2.2

3.3.1

Concordance

CQL

Concordance

CQL[word="[word=""][word=""][word=""][lemma=""][lemma=""]
"]

lemma
432
Collocation candidates••••••

162,975

Concordance

CQL[tag="V.*"][word="|WebCQL[word=""][word=""][lemma="1,170|

Collocation candidates"][word=""][lemma=""]"] [word=""] [lemma="10
Concordance"]

10,845[word="Collocation candidates

4,000

13(lexical sets)

1317"]

[word=""][word="|"][word=""][word=""] [word=""] [lemma=""]Srdanović 2007

Word Sketch

Word Sketch

3.3.

Sketch Engine
Sketch Engine

Sketch Engine
1)
Sketch Engineab
Sketch Engine

Sketch Engine

Nishina &Yoshihashi 2007Smrž 2004

18

Sketch Engine

2)
Sketch Engine

3)

a()

bcd

3.1

3.2

Smrž 2004

Sketch Engine
Sketch Difference

Thesaurus
Sketch Engine
Smrž

2004

Sketch Engine

Sketch Engine

4)

abc

Sketch Engine
Sketch Engine

Smith etal. 2007

19

3.4.
Sketch Engine
2.3

WebWebWord Sketch
Thesaurus

Joice 2005

Sketch Engine

ChaSenChaSenCorpus Builder

Sketch Engine

WebBootCatWeb

Baroni et al. 2006

4.
Sketch Engine

1)

ChaSen2)

4Web

ChaSenSketch Engine

Word Sketch Thesaurus Sketch Difference Concordance1) Web2)

3)

ChaSenChaSen20

Srdanović Erjavec, Irena

2007
19 , 83-89,

2007

Sketch Engine
18, 109-112,

2004
Baroni, Marko, Adam Kilgarriff, Jan Pomikalek & Pavel Rychly (2006) WebBootCaT: a web
tool for instant corpora, Proceedings of the EuraLex Conference 2006, 123-132.
Baroni, Marko & Silvia Bernardini, eds. (2006) Wacky! Working papers on the Web as Corpus,Bologna: GEDIT.
Church, Kenneth Ward & Patrick Hanks (1989) Word association norms, mutual information,and lexicography, Proceedings of the 27th annual meeting on Association for

Computational Linguistics, 76-83.
Crystal, David (2006) Language and the Internet, Cambridge: Cambridge University Press.
Erjavec, Tomaž, Kristina Hmeljak Sangawa & Irena Srdanović Erjavec (2006) jaSlo, A
Japanese-Slovene Learners' Dictionary: Methods for Dictionary Enhancement,

Proceedings of the 12th EURALEX International Congress
Erjavec, Tomaž, Adam Kilgarriff & Irena Srdanović Erjavec (2007) A large public-access
Japanese corpus and its query tool, CoJaS 2007, The Inaugural Workshop on

Computational Japanese Studies.
Gahl, Susanne (1998) Automatic Extraction of subcategorization frames for corpus-based
dictionary-building, Proc EURALEX 1998, 445-452.
Ghani, Rayid, Rosie Jones & Dunja Mladenic (2001) Using the Web to Create Minority
Language Corpora, Proceedings of the 2001 ACM CIKM: Tenth International

Conference on Information and Knowledge Management, 279-286.
Heid, Ulrich, Stefan Evert, Vincent Docherty, Wolfgang Worsch & Wermke, Matthias (2000)
Computational tools for semi-automatic corpus-based updating of dictionaries,EURALEX 2000 Proceedings, 183-196.
Joyce, Terry (2005) Constructing a large-scale database of Japanese word associations, In
Katsuo Tamaoka (ed.) Corpus Studies on Japanese Kanji (Glottometrics 10), 82-98,
Tokyo: Hituzi Syobo & Germany: RAM-Verlag:Ludenschied.
Keller, Frank & Maria Lapata (2003) Using the Web to Obtain Frequencies for Unseen
Bigrams, Computational Linguistics 29 (3), 459-484.
21

Kilgarriff, Adam & Michael Rundell (2002) Lexical Profiling Software and its Lexicographic
Applications - a Case Study, EURALEX 2002 Proceedings, 807-818.
Kilgarriff, Adam, Pavel Rychly, Pavel Smrž & David Tugwell (2004) The Sketch Engine, Proc.

Euralex, 105-116.
Kilgarriff Adam & David Tugwell (2001) WORD SKETCH: Extraction and Display ofSignificant

Collocations

forLexicography,Proc. workshop "COLLOCATION:

Computational Extraction, Analysis and Exploitation. 39th ACL & 10th EACL, 32-38.
Nishina, Kikuko & Kenji Yoshihashi (2007) Japanese Composition Support System
Displaying Occurrences and Example Sentences, Symposium on Large-scale

Knowledge Resources (LKR2007), 119-122.
Rundell, Michael, ed. (2002) Macmillan English Dictionary for Advanced Learners, London:
Macmillan.
Sharoff, Serge (2006) Open-source corpora: using the net to fish for linguistic data,

International Journal of Corpus Linguistics 11(4), 435-462.
Smith, Simon, Alice Chen & Adam Kilgarriff (2007) A corpus query tool for SLA: learning
Mandarin with the help of Sketch Engine, Practical Applications in Language andComputers - PALC 2007
Smrž, Pavel (2004) Integrating Natural Language Processing into E-learning — A Case ofCzech, Proceedings of the Workshop on eLearning for Computational Linguistics andComputational Linguistics for eLearning, COLING 2004. 106-111.
Srdanović Erjavec, Irena, Tomaž Erjavec & Adam Kilgarriff (2008) A web corpus and,word-sketches for Japanese,

Ueyama Motoko & Marko Baroni (2005) Automated construction and evaluation of aJapanese web-based reference corpus, Proceedings of Corpus Linguistics 2005.

22Sketch
Sketch Enginecorpus query tool for Japanese and its possible applications

SRDANOVIĆ ERJAVEC Irena, NISHINA Kikuko
Tokyo Institute of Technology

Keywords
Sketch Engine, corpus linguistics, lexicography, second language learning, collocations

Abstract

Although corpus-based language research has been developing rapidly in recent years,
there is still a lack of resources in regards to their size, textual variety, and time of creation,and of efficient and user-friendly corpus query tools. This is also the case for the Japanese
corpus linguistics, which is one of the primary reasons for the recent rise in projects
constructing Japanese corpora resources.
In this paper, we present a method for extracting linguistic information from corpora using
the Sketch Engine corpus query tool, which has recently been extended for the Japaneselanguage. The Japanese version is based on a 400 million word Japanese Web corpus, which
is linguistically annotated by the morphological analyzer ChaSen, and a Japanesegrammatical relations file. The tool offers efficient and user-friendly ways of extracting
concise linguistic data about words—their grammatical and collocational behavior, as well as
thesaurus-like information and differences in usage for similar words. We explain, throughexamples, how the tool could be utilized in corpus lexicography, linguistic research andcomputer assisted language learning of the Japanese language. The investigation part of thearticle concentrates mainly on the ways that the tool could be applied within the dictionary
creation process, and the results illustrate how each of the tool functions can greatly
contribute to that process.

23