ubiquity:
designing a multilingual
natural language interface
mitchoMichael Yoshitaka Erlewine

2009 SIGIR Workshop on Information
Access in a Multilingual World
Boston, July 23rd, 2009

ubiquity:
designing a multilingual natural language interface• focus on multilingual delivery and interfaces• reconsidering the text-based interface• localizing ubiquity

ubiquity:
designing a multilingual natural language interface• focus on multilingual delivery and interfaces• reconsidering the text-based interface• localizing ubiquityUSER

INFORMATIONINTERFACEUSER

INFORMATIONINTERFACE

EnglishDeutschEspañol
Français
Português
Русский
Italiano
Polski
Türkçe
日本語
中文

USER

INFORMATIONINTERFACE

USER

中文

INTERFACE

EnglishDeutschEspañol
Français
Português
Русский
Italiano
Polski
Türkçe日本語

INFORMATION

USER

中文

INTERFACE

EnglishDeutschEspañol
Français
Português
Русский
Italiano
Polski
Türkçe日本語

INFORMATION

this talkEQUIPOTENT MULTILINGUAL INTERFACES• not just translating content or localizing interfaces• ensuring equal access to information• ensuring equal generativity and potential of theplatform

EQUIPOTENT MULTILINGUAL INTERFACES• not just translating content or localizing interfaces• ensuring equal access to information• ensuring equal generativity and potential of theplatform

EQUIPOTENT MULTILINGUAL INTERFACES• not just translating content or localizing interfaces• ensuring equal access to information• ensuring equal generativity and potential of theplatform

“We create communities of people
involved in making the Internet
experience better for all of us... The
Internet is a global public resource thatmust remain open and accessible.”—Mozilla Manifesto

AfrikaansShqip‫ﻋﺮﺑﻲ‬অসমীয়া
EuskaraБеларуская
বাংলা (ভারত)Българскиcatalà
中文 (简体)正體中文 (繁體)HrvatskiČeština
Dansk
Nederlands
English (British)

English (US)
Esperanto
Eesti keel
suomi
Français
Frysk
Galegoქართული
DeutschΕλληνικά
ગ"જરાતી
‫עברית‬!ह#दी (भारत)Magyaríslenska
Bahasa Indonesia

Gaeilge
Italiano
日本語
Kannada한국어Kurdî
Latviešu
lietuvių kalbaМакедонски
मराठीNorsk bokmål
Norsk nynorsk
occitan (lengadocian)‫ﻓﺎﺭﺳﯽ‬PolskiPortuguês (do Brasil)

Português (Europeu)
ਪ"ਜਾਬੀ
română
Русский
српски
සිංහලslovenčina
slovensko
Español (de Argentina)
Español (de España)
Svenska
TeluguไทยУкраїнська
Tiếng ViệtCymraeg

ubiquity:
designing a multilingual natural language interface• focus on multilingual delivery and interfaces• reconsidering the text-based interface• localizing ubiquity

BUILDING A SCALABLE INTERFACE PARADIGMBUILDING A SCALABLE INTERFACE PARADIGM

The web is offering more and morefunctionality and API’s...

BUILDING A SCALABLE INTERFACE PARADIGM

The web is offering more and morefunctionality and API’s...
...but how do you use them?

BUILDING A SCALABLE INTERFACE PARADIGM

The web is offering more and morefunctionality and API’s...
...but how do you use them?
• Following the GUI paradigm...

BUILDING A SCALABLE INTERFACE PARADIGMThe web is offering more and morefunctionality and API’s...
...but how do you use them?
• Following the GUI paradigm...
• add more buttons?

BUILDING A SCALABLE INTERFACE PARADIGM

The web is offering more and morefunctionality and API’s...
...but how do you use them?
• Following the GUI paradigm...
• add more buttons?
• add more menus?

CC-BY-SA: flickr.com/photos/daveduarte/2678853923/

ACM interactions 2008

CC-BY: flickr.com/photos/striatic/1629254/

?

Aza Raskin on

text-based interfaces“...maybe this confusion isn’t the fault ofcommand-line interfaces in general; maybe
it’s just the command lines we’re used to. Ifcommands were memorable, and their syntax
forgiving, perhaps we wouldn’t be so scared
to reconsider these interface paradigms.”

CC-BY-SA: flickr.com/photos/
seanosh/3307290530/

Aza Raskin on

text-based interfaces“...maybe this confusion isn’t the fault ofcommand-line interfaces in general; maybe
it’s just the command lines we’re used to. Ifcommands were memorable, and their syntax
forgiving, perhaps we wouldn’t be so scared
to reconsider these interface paradigms.”

CC-BY-SA: flickr.com/photos/
seanosh/3307290530/

a natural language
interfaceubiquity
ubiquity.mozilla.com

[demo 1]
inserting a map into email(video not included in PDF)

without
ubiquity

without
ubiquity

without
ubiquity

without
ubiquity

[demo 1]
inserting a map into email:one step with ubiquity(video not included in PDF)

[demo 2]translating and emailing web content(video not included in PDF)

without
ubiquity

without
ubiquity

without
ubiquity

without
ubiquity

without
ubiquity

[demo 2]translating and emailing web content:
fast and straightforward with ubiquity(video not included in PDF)

goal:
how?

goal:

accomplish moreon the web

how?

goal:

accomplish moreon the web

how?

connect the web
with language

verbs**JavaScriptubiquity:
designing a multilingual natural language interface• focus on multilingual delivery and interfaces• reconsidering the text-based interface• localizing ubiquity

LOCALIZING UBIQUITY• The goal: “natural syntax”• Users must expect that not all
natural inputs areunderstandable.

• But the parser must not add

restrictions which go against theuser’s natural syntax.

LOCALIZING UBIQUITY• The goal: “natural syntax”• Users must expect that not all
natural inputs areunderstandable.

• But the parser must not add

restrictions which go against theuser’s natural syntax.

LOCALIZING UBIQUITY• EX: The following must behave equivalently:(Japanese, a scrambling language)• 太郎にボールを投げろ

Tarō-ni ball-o nager=oTarō-dat ball-acc throw=imper• ボールを太郎に投げろLOCALIZING UBIQUITY• EX: The following must behave equivalently:(Japanese)• ...投げろ nager=o (imperative)• ...投げる nager=u (infinitive)• ...投げて nage=te (linking form)

LOCALIZING UBIQUITY• Putting theoretical linguistic research andinsights into practice• Writing individual language parsers isdifficult and time-intensive• A Principles and Parameters model• One universal parser with minimal
language settings

LOCALIZING UBIQUITY• Putting theoretical linguistic research andinsights into practice• Writing individual language parsers isdifficult and time-intensive• A Principles and Parameters model• One universal parser with minimal
language settings

LOCALIZING UBIQUITY• Putting theoretical linguistic research andinsights into practice• Writing individual language parsers isdifficult and time-intensive• A Principles and Parameters model• One universal parser with minimal
language settings

LOCALIZING UBIQUITY• Putting theoretical linguistic research andinsights into practice• Writing individual language parsers isdifficult and time-intensive• A Principles and Parameters model• One universal parser with minimal
language settingsparser(animation not included in PDF)

parser(animation not included in PDF)

花子にケーキを送ってparser(animation not included in PDF)

花子にケーキを送ってparser(animation not included in PDF)

花子に ケーキを 送ってNNV

parser(animation not included in PDF)

花子に ケーキを 送ってN
goal

NobjectVparser(animation not included in PDF)

花子に ケーキを 送ってN
goal

NobjectV ( goal, object )VLOCALIZING UBIQUITY
Argument identification via semantic role
CmdUtils.CreateCommand({names: ["email", "mail"],arguments: [{role: "object", nountype: noun_arb_text},{role: "goal", nountype: noun_type_contact}],...

LOCALIZING UBIQUITY
Argument identification via semantic role
CmdUtils.CreateCommand({names: ["email", "mail"],arguments: [{role: "object", nountype: noun_arb_text},{role: "goal", nountype: noun_type_contact}],...

LOCALIZING UBIQUITY
Argument identification via semantic role

LOCALIZING UBIQUITY
Argument identification via semantic role

LOCALIZING UBIQUITY
Argument identification via semantic role
role

EnglishJapaneseobjectøをogoaltoに ni, へ elocation

at, onで deLOCALIZING UBIQUITY
Argument identification via semantic role
role

EnglishJapaneseobject

øをogoaltoに ni, へ elocationat, onで deLOCALIZING UBIQUITY
Argument identification via semantic role
role

EnglishJapaneseobject

øをogoaltoに ni, へ elocation

at, onで deLOCALIZING UBIQUITY
Argument identification via semantic role
role

EnglishJapaneseobject

øをogoaltoに ni, へ elocation

at, onで de[demo 3]
ubiquity in Japanese: “hello to Spanish”(video not included in PDF)

LOCALIZING UBIQUITY• Support for various linguistic structures• Equal support for verb-initial and-final languages• Smart argument-first suggestions

LOCALIZING UBIQUITY
Minimal language descriptions
branching: 'left',
usespaces: false,
joindelimiter: '',anaphora: ["これ", "それ", "あれ"],
roles: [{role: 'object', delimiter: 'を'},{role: 'goal', delimiter: 'に'},{role: 'source', delimiter: 'から'},{role: 'position', delimiter: 'で'},{role: 'position', delimiter: 'に'},{role: 'instrument', delimiter: 'で'},{role: 'alias', delimiter: 'として'},]

LOCALIZING UBIQUITY• An equipotent multilingual platform par excellence• Equivalent underlying parse structures• Commands via semantic role:
write once, use in any language• Easy to add new languages

LOCALIZING UBIQUITY• An equipotent multilingual platform par excellence• Equivalent underlying parse structures• Commands via semantic role:
write once, use in any language• Easy to add new languages

LOCALIZING UBIQUITY• An equipotent multilingual platform par excellence• Equivalent underlying parse structures• Commands via semantic role:
write once, use in any language• Easy to add new languages

ubiquity 0.5
with Parser 2

ubiquity 0.5
with Parser 2

Languagesettings:

Català
Dansk
EnglishEspañol
Français

Italiano
日本語
Português
Svenska
中文

ubiquity 0.5with Parser 2

ubiquity 0.5
with Parser 2

Català
Dansk
EnglishPortuguês
日本語

ubiquity:
designing a multilingual natural language interface• Conclusion• broad and equal access to information requires
equipotent multilingual interfaces• ubiquity: an equipotent platform par excellence• challenging internationalization possible
through application of linguistic researchubiquity:
designing a multilingual natural language interface• Conclusion• broad and equal access to information requires
equipotent multilingual interfaces• ubiquity: an equipotent platform par excellence• challenging internationalization possible
through application of linguistic researchubiquity:
designing a multilingual natural language interface• Conclusion• broad and equal access to information requires
equipotent multilingual interfaces• ubiquity: an equipotent platform par excellence• challenging internationalization possible
through application of linguistic researchThank you!
Questions?

mitchoMichael Yoshitaka Erlewine

by-nc-samitcho.com@mitchoyoshitaka