AI assistants have started carrying out tasks on a user's behalf by interacting directly with the web. However, training an interface that maps natural language (NL) commands to web actions is challenging for existing semantic parsing approaches due to the variable and unknown set of actions that characterize websites . We propose FLIN, a natural language interface for web navigation that maps NL commands to concept-level actions  rather than low-level UI interactions , thus being able to flexibly adapt to different websites and handle their transient nature. We frame this as a ranking problem where, given a user command and a webpage, FLIN learns to score the most appropriate navigation instruction (involving action and parameter values). To train and evaluate FLIN, we collect a dataset using nine popular websites from three different domains. Quantitative results show that FLIN is capable of adapting to new websites in a given domain.
AI assistants can now carry out tasks for users by directly interacting with website UIs. Current semantic parsing and slot-filling techniques cannot flexibly adapt to many different websites without being constantly re-trained . We propose FLIN, a natural language interface for web navigation that maps user commands to concept-level actions ( rather than low-level UI actions) , thus being able to flexibly adapt to different websites and handle their transient nature. We frame this as a ranking problem : given a user command and a webpage, FLIN learns to score the most relevant navigation instruction (involving action and parameter values). To train and evaluate FLIN, we collect a dataset using nine popular websites from three domains. Our results show that FLIN was able to adapt to new websites in a given domain.
R
have started carrying out tasks on a user's behalf by interacting directly with the web. However, training an interface that maps natural language (NL) commands to web actions is challenging for existing semantic parsing approaches due to the variable and unknown set of actions that characterize websites
can now carry out tasks for users by directly interacting with website UIs. Current semantic parsing and slot-filling techniques cannot flexibly adapt to many different websites without being constantly re-trained
R
NL
user
A
(
R
interactions
actions)
R
where,
:
R
appropriate
relevant
R
different domains. Quantitative
domains. Our
R
is capable of adapting
was able to adapt