The NLP Cypher | 12.20.20
By: Quantum Stat
Tags: Machine Learning, Deep Learning, Data Science, AI, Artificial Intelligence

NeurIPS & Knowledge Graphs

Michael Galkin returns with his round-up of graph news this time out of NeurIPS 🔥. Around five percent of papers from the conference were on graphs so lots to discuss.

His TOC:

Blog:

For an enterprise take w/r/t the power of knowledge graphs:

Training Data Extraction Attack 👀

A new paper (with authors from every major big tech), was recently published showing how one can attack language models like GPT-2 and extract information verbatim like personal identifiable information from just by querying the model. 🥶! The information extracted derived from the models’ training data that was based on scraped internet info. This is a big problem especially when you train a language model on a private custom dataset. The paper discusses causes and possible work arounds.

Paper

Book Me Some Data

Looks like Booking.com wants a new recommendation engine and they are offering up their dataset of over 1 million anonymized hotel reservations to get you in the game. Pretty cool if you want a chance in working with real-world data.

Here’s the training dataset schema:

user_id — User ID

check-in — Reservation check-in date

checkout — Reservation check-out date

affiliate_id — An anonymized ID of affiliate channels where the booker came from (e.g. direct, some third party referrals, paid search engine, etc.)

device_class — desktop/mobile

booker_country — Country from which the reservation was made (anonymized)

hotel_country — Country of the hotel (anonymized)

city_id — city_id of the hotel’s city (anonymized)

utrip_id — Unique identification of user’s trip (a group of multi-destinations bookings within the same trip)

“The eval dataset is similar to the train set except that the city_id of the final reservation of each trip is concealed and requires a prediction.”

UFO Files Dumped on Archive.org

There’s a nice dump of UFO files spanning several decades and countries if you want to get your alien research on. Apparently there was some copyright beef between content owners and media publishers which led ultimately to a third party obtaining a copy in the wild and uploading the files to archive.org 😭. Anyway, it’s a good data source to try out your latest OCR algo or if you are interested in searching for anti-gravity propulsion tech.

👽:

The Air Force Ported µZero

Apparently the US Air Force decided to port DeepMind’s µZero to the navigation/sensor system of a U-2 “dragon lady” spy plane. And they have called it ARTUµ, inspired by R2-D2 from Star Wars 😭. Recently, they ran the first ever simulated flight to show off the AI’s capabilities. The mission was for ARTUµ to conduct reconnaissance of enemy missile launchers on the ground while the pilot looked for aerial threats.

DARPA be like:

Article:

GitHub Search Index Update

GitHub will get rid of your repo from its code search index if it’s been inactive for more than a year. So how do you stay ‘active’?

“Recent activity for a repository means that it has had a commit or has shown up in a search result.”

Getting Rid of Intents

Alan Nichol opines on the latest state of conversational AI and his RASA platform with regards to the goal of getting rid of intents as being paramount for conversational AIs to achieve Kurzweil levels of robustness. They are currently experimenting with end-2-end learning as an alternative to intents.

In RASA 2.2 and beyond, intents will be optional.

Blog:

Speech Transformers & Datasets & WMT20 Model Checkpoints from FAIR

XLSR-53: Multilingual Self-Supervised Speech Transformer

Multilingual pre-trained wav2vec 2.0 models

Multi-Lingual LibriSpeech Dataset

WMT Models Out

Facebook FAIR’s WMT’20 news translation task submission models

Repo Cypher 👨‍💻

A collection of recently released repos that caught our 👁