prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What limitations do the authors demnostrate of their model?
Context 1: Introduction
Neural networks have been successfully used to describe images with text using sequence-to-sequence models BIBREF0  However, the results are simple and dry captions which are one or two phrases long  Humans looking at a painting see more than just objects  Paintings stimulate sentiments, metaphors and stories as well  Therefore, our goal is to have a neural network describe the painting artistically in a style of choice  As a proof of concept, we present a model which generates Shakespearean prose for a given painting as shown in Figure FIGREF1  Accomplishing this task is difficult with traditional sequence to sequence models since there does not exist a large collection of Shakespearean prose which describes paintings: Shakespeare's works describes a single painting shown in Figure FIGREF3  Fortunately we have a dataset of modern English poems which describe images BIBREF1 and line-by-line modern paraphrases of Shakespeare's plays BIBREF2  Our solution is therefore to combine two separately trained models to synthesize Shakespearean prose for a given painting 
Introduction ::: Related work
A general end-to-end approach to sequence learning BIBREF3 makes minimal assumptions on the sequence structure  This model is widely used in tasks such as machine translation, text summarization, conversational modeling, and image captioning  A generative model using a deep recurrent architecture BIBREF0 has also beeen used for generating phrases describing an image  The task of synthesizing multiple lines of poetry for a given image BIBREF1 is accomplished by extracting poetic clues from images  Given the context image, the network associates image attributes with poetic descriptions using a convolutional neural net  The poem is generated using a recurrent neural net which is trained using multi-adversarial training via policy gradient 
Transforming text from modern English to Shakespearean English using text "style transfer" is challenging  An end to end approach using a sequence-to-sequence model over a parallel text corpus BIBREF2 has been proposed based on machine translation  In the absence of a parallel text corpus, generative adversarial networks (GANs) have been used, which simultaneously train two models: a generative model which captures the data distribution, and a discriminative model which evaluates the performance of the generator  Using a target domain language model as a discriminator has also been employed BIBREF4, providing richer and more stable token-level feedback during the learning process  A key challenge in both image and text style transfer is separating content from style BIBREF5, BIBREF6, BIBREF7  Cross-aligned auto-encoder models have focused on style transfer using non-parallel text BIBREF7  Recently, a fine grained model for text style transfer has been proposed BIBREF8 which controls several factors of variation in textual data by using back-translation  This allows control over multiple attributes, such as gender and sentiment, and fine-grained control over the trade-off between content and style 
Methods
We use a total three datasets: two datasets for generating an English poem from an image, and Shakespeare plays and their English translations for text style transfer 
We train a model for generating poems from images based on two datasets BIBREF1  The first dataset consists of image and poem pairs, namely a multi-modal poem dataset (MultiM-Poem), and the second dataset is a large poem corpus, namely a uni-modal poem dataset (UniM-Poem)  The image and poem pairs are extended by adding the nearest three neighbor poems from the poem corpus without redundancy, and an extended image and poem pair dataset is constructed and denoted as MultiM-Poem(Ex)BIBREF1 
We use a collection of line-by-line modern paraphrases for 16 of Shakespeare’s plays BIBREF2, for training a style transfer network from English poems to Shakespearean prose  We use 18,395 sentences from the training data split  We keep 1,218 sentences in the validation data set and 1,462 sentences in our test set 
Methods ::: Image To Poem Actor-Critic Model
For generating a poem from images we use an existing actor-critic architecture BIBREF1  This involves 3 parallel CNNs: an object CNN, sentiment CNN, and scene CNN, for feature extraction  These features are combined with a skip-thought model which provides poetic clues, which are then fed into a sequence-to-sequence model trained by policy gradient with 2 discriminator networks for rewards  This as a whole forms a pipeline that takes in an image and outputs a poem as shown on the top left of Figure FIGREF4  A CNN-RNN generative model acts as an agent  The parameters of this agent define a policy whose execution determines which word is selected as an action  When the agent selects all words in a poem, it receives a reward  Two discriminative networks, shown on the top right of Figure FIGREF4, are defined to serve as rewards concerning whether the generated poem properly describes the input image and whether the generated poem is poetic  The goal of the poem generation model is to generate a sequence of words as a poem for an image to maximize the expected return 
Methods ::: Shakespearizing Poetic Captions
For Shakespearizing modern English texts, we experimented with various types of sequence to sequence models  Since the size of the parallel translation data available is small, we leverage a dictionary providing a mapping between Shakespearean words and modern English words to retrofit pre-trained word embeddings  Incorporating this extra information improves the translation task  The large number of shared word types between the source and target sentences indicates that sharing the representation between them is beneficial 
Methods ::: Shakespearizing Poetic Captions ::: Seq2Seq with Attention
We use a sequence-to-sequence model which consists of a single layer unidrectional LSTM encoder and a single layer LSTM decoder and pre-trained retrofitted word embeddings shared between source and target sentences  We experimented with two different types of attention: global attention BIBREF9, in which the model makes use of the output from the encoder and decoder for the current time step only, and Bahdanau attention BIBREF10, where computing attention requires the output of the decoder from the prior time step  We found that global attention performs better in practice for our task of text style transfer 
Methods ::: Shakespearizing Poetic Captions ::: Seq2Seq with a Pointer Network
Since a pair of corresponding Shakespeare and modern English sentences have significant vocabulary overlap we extend the sequence-to-sequence model mentioned above using pointer networks BIBREF11 that provide location based attention and have been used to enable copying of tokens directly from the input  Moreover, there are lot of proper nouns and rare words which might not be predicted by a vanilla sequence to sequence model 
Methods ::: Shakespearizing Poetic Captions ::: Prediction
For both seq2seq models, we use the attention matrices returned at each decoder time step during inference, to compute the next word in the translated sequence if the decoder output at the current time step is the UNK token  We replace the UNKs in the target output with the highest aligned, maximum attention, source word  The seq2seq model with global attention gives the best results with an average target BLEU score of 29 65 on the style transfer dataset, compared with an average target BLEU score of 26 97 using the seq2seq model with pointer networks 
Results
We perform a qualitative analysis of the Shakespearean prose generated for the input paintings  We conducted a survey, in which we presented famous paintings including those shown in Figures FIGREF1 and FIGREF10 and the corresponding Shakespearean prose generated by the model, and asked 32 students to rate them on the basis of content, creativity and similarity to Shakespearean style on a Likert scale of 1-5  Figure FIGREF12 shows the result of our human evaluation 
The average content score across the paintings is 3 7 which demonstrates that the prose generated is relevant to the painting  The average creativity score is 3 9 which demonstrates that the model captures more than basic objects in the painting successfully using poetic clues in the scene  The average style score is 3 9 which demonstrates that the prose generated is perceived to be in the style of Shakespeare 
We also perform a quantitative analysis of style transfer by generating BLEU scores for the model output using the style transfer dataset  The variation of the BLEU scores with the source sentence lengths is shown in Figure FIGREF11  As expected, the BLEU scores decrease with increase in source sentence lengths 
Results ::: Implementation
All models were trained on Google Colab with a single GPU using Python 3 6 and Tensorflow 2 0  The number of hidden units for the encoder and decoder is 1,576 and 256 for seq2seq with global attention and seq2seq with pointer networks respectively  Adam optimizer was used with the default learning rate of 0 001  The model was trained for 25 epochs  We use pre-trained retrofitted word embeddings of dimension 192 
Results ::: Limitations
Since we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer as shown in Figure FIGREF12 for "Starry Night" with a low average content score  This happens when the style transfer dataset does not have similar words in the training set of sentences  A solution would be to expand the style transfer dataset, for a better representation of the poem data 
Results ::: Conclusions and Future Work
In conclusion, combining two pipelines with an intermediate representation works well in practice  We observe that a CNN-RNN based image-to-poem net combined with a seq2seq model with parallel text corpus for text style transfer synthesizes Shakespeare-style prose for a given painting  For the seq2seq model used, we observe that it performs better in practice using global attention as compared with local attention  We make our models and code publicly available BIBREF12  In future work we would like to experiment with GANs in the absence of non-parallel datasets, so that we can use varied styles for text style transfer  We would also like to experiment with cross aligned auto-encoders, which form a latent content representation, to efficiently separate style and content 
Context 2: Introduction
Twitter is a widely used microblogging platform, where users post and interact with messages, “tweets”  Understanding the semantic representation of tweets can benefit a plethora of applications such as sentiment analysis BIBREF0 , BIBREF1 , hashtag prediction BIBREF2 , paraphrase detection BIBREF3 and microblog ranking BIBREF4 , BIBREF5   However, tweets are difficult to model as they pose several challenges such as short length, informal words, unusual grammar and misspellings  Recently, researchers are focusing on leveraging unsupervised representation learning methods based on neural networks to solve this problem  Once these representations are learned, we can use off-the-shelf predictors taking the representation as input to solve the downstream task BIBREF6 , BIBREF7   These methods enjoy several advantages: (1) they are cheaper to train, as they work with unlabelled data, (2) they reduce the dependence on domain level experts, and (3) they are highly effective across multiple applications, in practice 
Despite this, there is a lack of prior work which surveys the tweet-specific unsupervised representation learning models  In this work, we attempt to fill this gap by investigating the models in an organized fashion  Specifically, we group the models based on the objective function it optimizes  We believe this work can aid the understanding of the existing literature  We conclude the paper by presenting interesting future research directions, which we believe are fruitful in advancing this field by building high-quality tweet representation learning models 
Unsupervised Tweet Representation Models
There are various models spanning across different model architectures and objective functions in the literature to compute tweet representation in an unsupervised fashion  These models work in a semi-supervised way - the representations generated by the model is fed to an off-the-shelf predictor like Support Vector Machines (SVM) to solve a particular downstream task  These models span across a wide variety of neural network based architectures including average of word vectors, convolutional-based, recurrent-based and so on  We believe that the performance of these models is highly dependent on the objective function it optimizes – predicting adjacent word (within-tweet relationships), adjacent tweet (inter-tweet relationships), the tweet itself (autoencoder), modeling from structured resources like paraphrase databases and weak supervision  In this section, we provide the first of its kind survey of the recent tweet-specific unsupervised models in an organized fashion to understand the literature  Specifically, we categorize each model based on the optimized objective function as shown in Figure FIGREF1   Next, we study each category one by one 
Modeling within-tweet relationships
Motivation: Every tweet is assumed to have a latent topic vector, which influences the distribution of the words in the tweet  For example, though the appearance of the phrase catch the ball is frequent in the corpus, if we know that the topic of a tweet is about “technology”, we can expect words such as bug or exception after the word catch (ignoring the) instead of the word ball since catch the bug/exception is more plausible under the topic “technology”  On the other hand, if the topic of the tweet is about “sports”, then we can expect ball after catch  These intuitions indicate that the prediction of neighboring words for a given word strongly relies on the tweet also 
Models: BIBREF8 's work is the first to exploit this idea to compute distributed document representations that are good at predicting words in the document  They propose two models: PV-DM and PV-DBOW, that are extensions of Continuous Bag Of Words (CBOW) and Skip-gram model variants of the popular Word2Vec model BIBREF9 respectively – PV-DM inserts an additional document token (which can be thought of as another word) which is shared across all contexts generated from the same document; PV-DBOW attempts to predict the sampled words from the document given the document representation  Although originally employed for paragraphs and documents, these models work better than the traditional models: BOW BIBREF10 and LDA BIBREF11 for tweet classification and microblog retrieval tasks BIBREF12   The authors in BIBREF12 make the PV-DM and PV-DBOW models concept-aware (a rich semantic signal from a tweet) by augmenting two features: attention over contextual words and conceptual tweet embedding, which jointly exploit concept-level senses of tweets to compute better representations  Both the discussed works have the following characteristics: (1) they use a shallow architecture, which enables fast training, (2) computing representations for test tweets requires computing gradients, which is time-consuming for real-time Twitter applications, and (3) most importantly, they fail to exploit textual information from related tweets that can bear salient semantic signals 
Modeling inter-tweet relationships
Motivation: To capture rich tweet semantics, researchers are attempting to exploit a type of sentence-level Distributional Hypothesis BIBREF10 , BIBREF13   The idea is to infer the tweet representation from the content of adjacent tweets in a related stream like users' Twitter timeline, topical, retweet and conversational stream  This approach significantly alleviates the context insufficiency problem caused due to the ambiguous and short nature of tweets BIBREF0 , BIBREF14  
Models: Skip-thought vectors BIBREF15 (STV) is a widely popular sentence encoder, which is trained to predict adjacent sentences in the book corpus BIBREF16   Although the testing is cheap as it involves a cheap forward propagation of the test sentence, STV is very slow to train thanks to its complicated model architecture  To combat this computational inefficiency, FastSent BIBREF17 propose a simple additive (log-linear) sentence model, which predicts adjacent sentences (represented as BOW) taking the BOW representation of some sentence in context  This model can exploit the same signal, but at a much lower computational expense  Parallel to this work, Siamase CBOW BIBREF18 develop a model which directly compares the BOW representation of two sentence to bring the embeddings of a sentence closer to its adjacent sentence, away from a randomly occurring sentence in the corpus  For FastSent and Siamese CBOW, the test sentence representation is a simple average of word vectors obtained after training  Both of these models are general purpose sentence representation models trained on book corpus, yet give a competitive performance over previous models on the tweet semantic similarity computation task  BIBREF14 's model attempt to exploit these signals directly from Twitter  With the help of attention technique and learned user representation, this log-linear model is able to capture salient semantic information from chronologically adjacent tweets of a target tweet in users' Twitter timeline 
Modeling from structured resources
Motivation: In recent times, building representation models based on supervision from richly structured resources such as Paraphrase Database (PPDB) BIBREF19 (containing noisy phrase pairs) has yielded high quality sentence representations  These methods work by maximizing the similarity of the sentences in the learned semantic space 
Models: CHARAGRAM BIBREF20 embeds textual sequences by learning a character-based compositional model that involves addition of the vectors of its character n-grams followed by an elementwise nonlinearity  This simpler architecture trained on PPDB is able to beat models with complex architectures like CNN, LSTM on SemEval 2015 Twitter textual similarity task by a large margin  This result emphasizes the importance of character-level models that address differences due to spelling variation and word choice  The authors in their subsequent work BIBREF21 conduct a comprehensive analysis of models spanning the range of complexity from word averaging to LSTMs for its ability to do transfer and supervised learning after optimizing a margin based loss on PPDB  For transfer learning, they find models based on word averaging perform well on both the in-domain and out-of-domain textual similarity tasks, beating LSTM model by a large margin  On the other hand, the word averaging models perform well for both sentence similarity and textual entailment tasks, outperforming the LSTM  However, for sentiment classification task, they find LSTM (trained on PPDB) to beat the averaging models to establish a new state of the art  The above results suggest that structured resources play a vital role in computing general-purpose embeddings useful in downstream applications 
Modeling as an autoencoder
Motivation: The autoencoder based approach learns latent (or compressed) representation by reconstructing its own input  Since textual data like tweets contain discrete input signals, sequence-to-sequence models BIBREF22 like STV can be used to build the solution  The encoder model which encodes the input tweet can typically be a CNN BIBREF23 , recurrent models like RNN, GRU, LSTM BIBREF24 or memory networks BIBREF25   The decoder model which generates the output tweet can typically be a recurrent model that predicts a output token at every time step 
Models: Sequential Denoising Autoencoders (SDAE) BIBREF17 is a LSTM-based sequence-to-sequence model, which is trained to recover the original data from the corrupted version  SDAE produces robust representations by learning to represent the data in terms of features that explain its important factors of variation  Tweet2Vec BIBREF3 is a recent model which uses a character-level CNN-LSTM encoder-decoder architecture trained to construct the input tweet directly  This model outperforms competitive models that work on word-level like PV-DM, PV-DBOW on semantic similarity computation and sentiment classification tasks, thereby showing that the character-level nature of Tweet2Vec is best-suited to deal with the noise and idiosyncrasies of tweets  Tweet2Vec controls the generalization error by using a data augmentation technique, wherein tweets are replicated and some of the words in the replicated tweets are replaced with their synonyms  Both SDAE and Tweet2Vec has the advantage that they don't need a coherent inter-sentence narrative (like STV), which is hard to obtain in Twitter 
Modeling using weak supervision
Motivation: In a weakly supervised setup, we create labels for a tweet automatically and predict them to learn potentially sophisticated models than those obtained by unsupervised learning alone  Examples of labels include sentiment of the overall tweet, words like hashtag present in the tweet and so on  This technique can create a huge labeled dataset especially for building data-hungry, sophisticated deep learning models 
Models: BIBREF26 learns sentiment-specific word embedding (SSWE), which encodes the polarity information in the word representations so that words with contrasting polarities and similar syntactic context (like good and bad) are pushed away from each other in the semantic space that it learns  SSWE utilizes the massive distant-supervised tweets collected by positive and negative emoticons to build a powerful tweet representation, which are shown to be useful in tasks such as sentiment classification and word similarity computation in sentiment lexicon  BIBREF2 observes that hashtags in tweets can be considered as topics and hence tweets with similar hashtags must come closer to each other  Their model predicts the hashtags by using a Bi-GRU layer to embed the tweets from its characters  Due to subword modeling, such character-level models can approximate the representations for rare words and new words (words not seen during training) in the test tweets really well  This model outperforms the word-level baselines for hashtag prediction task, thereby concluding that exploring character-level models for tweets is a worthy research direction to pursue  Both these works fail to study the model's generality BIBREF27 , i e , the ability of the model to transfer the learned representations to diverse tasks 
Future Directions
In this section we present the future research directions which we believe can be worth pursuing to generate high quality tweet embeddings 
Conclusion
In this work we study the problem of learning unsupervised tweet representations  We believe our survey of the existing works based on the objective function can give vital perspectives to researchers and aid their understanding of the field  We also believe the future research directions studied in this work can help in breaking the barriers in building high quality, general purpose tweet representation models 
Context 3: Introduction
Users of photo-sharing websites such as Flickr often provide short textual descriptions in the form of tags to help others find the images  With the availability of GPS systems in current electronic devices such as smartphones, latitude and longitude coordinates are nowadays commonly made available as well  The tags associated with such georeferenced photos often describe the location where these photos were taken, and Flickr can thus be regarded as a source of environmental information  The use of Flickr for modelling urban environments has already received considerable attention  For instance, various approaches have been proposed for modelling urban regions BIBREF0 , and for identifying points-of-interest BIBREF1 and itineraries BIBREF2 , BIBREF3   However, the usefulness of Flickr for characterizing the natural environment, which is the focus of this paper, is less well-understood 
Many recent studies have highlighted that Flickr tags capture valuable ecological information, which can be used as a complementary source to more traditional sources  To date, however, ecologists have mostly used social media to conduct manual evaluations of image content with little automated exploitation of the associated tags BIBREF4 , BIBREF5 , BIBREF6   One recent exception is BIBREF7 , where bag-of-words representations derived from Flickr tags were found to give promising result for predicting a range of different environemental phenomena 
Our main hypothesis in this paper is that by using vector space embeddings instead of bag-of-words representations, the ecological information which is implicitly captured by Flickr tags can be utilized in a more effective way  Vector space embeddings are representations in which the objects from a given domain are encoded using relatively low-dimensional vectors  They have proven useful in natural language processing, especially for encoding word meaning BIBREF8 , BIBREF9 , and in machine learning more generally  In this paper, we are interested in the use of such representations for modelling geographic locations  Our main motivation for using vector space embeddings is that they allow us to integrate the textual information we get from Flickr with available structured information in a very natural way  To this end, we rely on an adaptation of the GloVe word embedding model BIBREF9 , but rather than learning word vectors, we learn vectors representing locations  Similar to how the representation of a word in GloVe is determined by the context words surrounding it, the representation of a location in our model is determined by the tags of the photos that have been taken near that location  To incorporate numerical features from structured environmental datasets (e g  average temperature), we associate with each such feature a linear mapping that can be used to predict that feature from a given location vector  This is inspired by the fact that salient properties of a given domain can often be modelled as directions in vector space embeddings BIBREF10 , BIBREF11 , BIBREF12   Finally, evidence from categorical datasets (e g  land cover types) is taken into account by requiring that locations belonging to the same category are represented using similar vectors, similar to how semantic types are sometimes modelled in the context of knowledge graph embedding BIBREF13  
While our point-of-departure is a standard word embedding model, we found that the off-the-shelf GloVe model performed surprisingly poorly, meaning that a number of modifications are needed to achieve good results  Our main findings are as follows  First, given that the number of tags associated with a given location can be quite small, it is important to apply some kind of spatial smoothing, i e  the importance of a given tag for a given location should not only depend on the occurrences of the tag at that location, but also on its occurrences at nearby locations  To this end, we use a formulation which is based on spatially smoothed version of pointwise mutual information  Second, given the wide diversity in the kind of information that is covered by Flickr tags, we find that term selection is in some cases critical to obtain vector spaces that capture the relevant aspects of geographic locations  For instance, many tags on Flickr refer to photography related terms, which we would normally not want to affect the vector representation of a given location  Finally, even with these modifications, vector space embeddings learned from Flickr tags alone are sometimes outperformed by bag-of-words representations  However, our vector space embeddings lead to substantially better predictions in cases where structured (scientific) information is also taken into account  In this sense, the main value of using vector space embeddings in this context is not so much about abstracting away from specific tag usages, but rather about the fact that such representations allow us to integrate numerical and categorical features in a much more natural way than is possible with bag-of-words representations 
The remainder of this paper is organized as follows  In the next section, we provide a discussion of existing work  Section SECREF3 then presents our model for embedding geographic locations from Flickr tags and structured data  Next, in Section SECREF4 we provide a detailed discussion about the experimental results  Finally, Section SECREF5 summarizes our conclusions 
Vector space embeddings
The use of low-dimensional vector space embeddings for representing objects has already proven effective in a large number of applications, including natural language processing (NLP), image processing, and pattern recognition  In the context of NLP, the most prominent example is that of word embeddings, which represent word meaning using vectors of typically around 300 dimensions  A large number of different methods for learning such word embeddings have already been proposed, including Skip-gram and the Continuous Bag-of-Words (CBOW) model BIBREF8 , GloVe BIBREF9 , and fastText BIBREF14   They have been applied effectively in many downstream NLP tasks such as sentiment analysis BIBREF15 , part of speech tagging BIBREF16 , BIBREF17 , and text classification BIBREF18 , BIBREF19   The model we consider in this paper builds on GloVe, which was designed to capture linear regularities of word-word co-occurrence  In GloVe, there are two word vectors INLINEFORM0 and INLINEFORM1 for each word in the vocabulary, which are learned by minimizing the following objective: DISPLAYFORM0
where INLINEFORM0 is the number of times that word INLINEFORM1 appears in the context of word INLINEFORM2 , INLINEFORM3 is the vocabulary size, INLINEFORM4 is the target word bias, INLINEFORM5 is the context word bias  The weighting function INLINEFORM6 is used to limit the impact of rare terms  It is defined as 1 if INLINEFORM7 and as INLINEFORM8 otherwise, where INLINEFORM9 is usually fixed to 100 and INLINEFORM10 to 0 75  Intuitively, the target word vectors INLINEFORM11 correspond to the actual word representations which we would like to find, while the context word vectors INLINEFORM12 model how occurrences of INLINEFORM13 in the context of a given word INLINEFORM14 affect the representation of this latter word  In this paper we will use a similar model, which will however be aimed at learning location vectors instead of the target word vectors 
Beyond word embeddings, various methods have been proposed for learning vector space representations from structured data such as knowledge graphs BIBREF20 , BIBREF21 , BIBREF22 , social networks BIBREF23 , BIBREF24 and taxonomies BIBREF25 , BIBREF26   The idea of combining a word embedding model with structured information has also been explored by several authors, for example to improve the word embeddings based on information coming from knowledge graphs BIBREF27 , BIBREF28   Along similar lines, various lexicons have been used to obtain word embeddings that are better suited at modelling sentiment BIBREF15 and antonymy BIBREF29 , among others  The method proposed by BIBREF30 imposes the condition that words that belong to the same semantic category are closer together than words from different categories, which is somewhat similar in spirit to how we will model categorical datasets in our model 
Embeddings for geographic information
The problem of representing geographic locations using embeddings has also attracted some attention  An early example is BIBREF31 , which used principal component analysis and stacked autoencoders to learn low-dimensional vector representations of city neighbourhoods based on census data  They use these representations to predict attributes such as crime, which is not included in the given census data, and find that in most of the considered evaluation tasks, the low-dimensional vector representations lead to more faithful predictions than the original high-dimensional census data 
Some existing works combine word embedding models with geographic coordinates  For example, in BIBREF32 an approach is proposed to learn word embeddings based on the assumption that words which tend to be used in the same geographic locations are likely to be similar  Note that their aim is dual to our aim in this paper: while they use geographic location to learn word vectors, we use textual descriptions to learn vectors representing geographic locations 
Several methods also use word embedding models to learn representations of Points-of-Interest (POIs) that can be used for predicting user visits BIBREF33 , BIBREF34 , BIBREF35   These works use the machinery of existing word embedding models to learn POI representations, intuitively by letting sequences of POI visits by a user play the role of sequences of words in a sentence  In other words, despite the use of word embedding models, many of these approaches do not actually consider any textual information  For example, in BIBREF34 the Skip-gram model is utilized to create a global pattern of users' POIs  Each location was treated as a word and the other locations visited before or after were treated as context words  They then use a pair-wise ranking loss BIBREF36 which takes into account the user's location visit frequency to personalize the location recommendations  The methods of BIBREF34 were extended in BIBREF35 to use a temporal embedding and to take more account of geographic context, in particular the distances between preferred and non-preferred neighboring POIs, to create a “geographically hierarchical pairwise preference ranking model”  Similarly, in BIBREF37 the CBOW model was trained with POI data  They ordered POIs spatially within the traffic-based zones of urban areas  The ordering was used to generate characteristic vectors of POI types  Zone vectors represented by averaging the vectors of the POIs contained in them, were then used as features to predict land use types  In the CrossMap method BIBREF38 they learned embeddings for spatio-temporal hotspots obtained from social media data of locations, times and text  In one form of embedding, intended to enable reconstruction of records, neighbourhood relations in space and time were encoded by averaging hotspots in a target location's spatial and temporal neighborhoods  They also proposed a graph-based embedding method with nodes of location, time and text

ground_truth: Since we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer

we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer as shown in Figure FIGREF12 for "Starry Night" with a low average content score

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which dataset do they use?
Context 3:  The concatenation of the location, time and text vectors were then used as features to predict peoples' activities in urban environments  Finally, in BIBREF39 , a method is proposed that uses the Skip-gram model to represent POI types, based on the intuition that the vector representing a given POI type should be predictive of the POI types that found near places of that type 
Our work is different from these studies, as our focus is on representing locations based on a given text description of that location (in the form of Flickr tags), along with numerical and categorical features from scientific datasets 
Analyzing Flickr tags
Many studies have focused on analyzing Flickr tags to extract useful information in domains such as linguistics BIBREF40 , geography BIBREF0 , BIBREF41 , and ecology BIBREF42 , BIBREF7 , BIBREF43   Most closely related to our work, BIBREF7 found that the tags of georeferenced Flickr photos can effectively supplement traditional scientific environmental data in tasks such as predicting climate features, land cover, species occurrence, and human assessments of scenicness  To encode locations, they simply combine a bag-of-words representation of geographically nearby tags with a feature vector that encodes associated structured scientific data  They found that the predictive value of Flickr tags is roughly on a par with that of the scientific datasets, and that combining both types of information leads to significantly better results than using either of them alone  As we show in this paper, however, their straightforward way of combining both information sources, by concatenating the two types of feature vectors, is far from optimal 
Despite the proven importance of Flickr tags, the problem of embedding Flickr tags has so far received very limited attention  To the best of our knowledge, BIBREF44 is the only work that generated embeddings for Flickr tags  However, their focus was on learning embeddings that capture word meaning (being evaluated on word similarity tasks), whereas we use such embeddings as part of our method for representing locations 
Model Description
In this section, we introduce our embedding model, which combines Flickr tags and structured scientific information to represent a set of locations INLINEFORM0   The proposed model has the following form: DISPLAYFORM0
where INLINEFORM0 and INLINEFORM1 are parameters to control the importance of each component in the model  Component INLINEFORM2 will be used to constrain the representation of the locations based on their textual description (i e  Flickr tags), INLINEFORM3 will be used to constrain the representation of the locations based on their numerical features, and INLINEFORM4 will impose the constraint that locations belonging to the same category should be close together in the space  We will discuss each of these components in more detail in the following sections 
Tag Based Location Embedding
Many of the tags associated with Flickr photos describe characteristics of the places where these photos were taken BIBREF45 , BIBREF46 , BIBREF47   For example, tags may correspond to place names (e g  Brussels, England, Scandinavia), landmarks (e g  Eiffel Tower, Empire State Building) or land cover types (e g  mountain, forest, beach)  To allow us to build location models using such tags, we collected the tags and meta-data of 70 million Flickr photos with coordinates in Europe (which is the region our experiments will focus on), all of which were uploaded to Flickr before the end of September 2015  In this section we first explain how tags can be weighted to obtain bag-of-words representations of locations from Flickr  Subsequently we describe a tag selection method, which will allow us to specialize the embedding depending on which aspects of the considered locations are of interest, after which we discuss the actual embedding model 
Tag weighting  Let INLINEFORM0 be a set of geographic locations, each characterized by latitude and longitude coordinates  To generate a bag-of-words representation of a given location, we have to weight the relevance of each tag to that location  To this end, we have followed the weighting scheme from BIBREF7 , which combines a Gaussian kernel (to model spatial proximity) with Positive Pointwise Mutual Information (PPMI) BIBREF48 , BIBREF49  
Let us write INLINEFORM0 for the set of users who have assigned tag INLINEFORM1 to a photo with coordinates near INLINEFORM2   To assess how relevant INLINEFORM3 is to the location INLINEFORM4 , the number of times INLINEFORM5 occurs in photos near INLINEFORM6 is clearly an important criterion  However, rather than simply counting the number of occurrences within some fixed radius, we use a Gaussian kernel to weight the tag occurrences according to their distance from that location: INLINEFORM7
where the threshold INLINEFORM0 is assumed to be fixed, INLINEFORM1 is the location of a Flickr photo, INLINEFORM2 is the Haversine distance, and we will assume that the bandwidth parameter INLINEFORM3 is set to INLINEFORM4   A tag occurrence is counted only once for all photos by the same user at the same location, which is important to reduce the impact of bulk uploading  The value INLINEFORM5 reflects how frequent tag INLINEFORM6 is near location INLINEFORM7 , but it does not yet take into account the total number of tag occurrences near INLINEFORM8 , nor how popular the tag INLINEFORM9 is overall  To measure how strongly tag INLINEFORM10 is associated with location INLINEFORM11 , we use PPMI, which is a commonly used measure of association in natural language processing  However, rather than estimating PPMI scores from term frequencies, we will use the INLINEFORM12 values instead: INLINEFORM13
where: INLINEFORM0
with INLINEFORM0 the set of all tags, and INLINEFORM1 the set of locations 
Tag selection  Inspired by BIBREF50 , we use a term selection method in order to focus on the tags that are most important for the tasks that we want to consider and reduce the impact of tags that might relate only to a given individual or a group of users  In particular, we obtained good results with a method based on Kullback-Leibler (KL) divergence, which is based on BIBREF51   Let INLINEFORM0 be a set of (mutually exclusive) properties of locations in which we are interested (e g  land cover categories)  For the ease of presentation, we will identify INLINEFORM1 with the set of locations that have the corresponding property  Then, we select tags from INLINEFORM2 that maximize the following score: INLINEFORM3
where INLINEFORM0 is the probability that a photo with tag INLINEFORM1 has a location near INLINEFORM2 and INLINEFORM3 is the probability that an arbitrary tag occurrence is assigned to a photo near a location in INLINEFORM4   Since INLINEFORM5 often has to be estimated from a small number of tag occurrences, it is estimated using Bayesian smoothing: INLINEFORM6
where INLINEFORM0 is a parameter controlling the amount of smoothing, which will be tuned in the experiments  On the other hand, for INLINEFORM1 we can simply use a maximum likelihood estimation: INLINEFORM2
Location embedding  We now want to find a vector INLINEFORM0 for each location INLINEFORM1 such that similar locations are represented using similar vectors  To achieve this, we use a close variant of the GloVe model, where tag occurrences are treated as context words of geographic locations  In particular, with each location INLINEFORM2 we associate a vector INLINEFORM3 and with each tag INLINEFORM4 we associate a vector INLINEFORM5 and a bias term INLINEFORM6 , and consider the following objective (which in our full model ( EQREF7 ) will be combined with components that are derived from the structured information): INLINEFORM7
Note how tags play the role of the context words in the GloVe model, while instead of learning target word vectors we now learn location vectors  In contrast to GloVe, our objective does not directly refer to co-occurrence statistics, but instead uses the INLINEFORM0 scores  One important consequence of this is that we can also consider pairs INLINEFORM1 for which INLINEFORM2 does not occur in INLINEFORM3 at all; such pairs are usually called negative examples  While they cannot be used in the standard GloVe model, some authors have already reported that introducing negative examples in variants of GloVe can lead to an improvement BIBREF52   In practice, evaluating the full objective above would not be computationally feasible, as we may need to consider millions of locations and millions of tags  Therefore, rather than considering all tags in INLINEFORM4 for the inner summation, we only consider those tags that appear at least once near location INLINEFORM5 together with a sample of negative examples 
Structured Environmental Data
There is a wide variety of structured data that can be used to describe locations  In this work, we have restricted ourselves to the same datasets as BIBREF7   These include nine (real-valued) numerical features, which are latitude, longitude, elevation, population, and five climate related features (avg  temperature, avg  precipitation, avg  solar radiation, avg  wind speed, and avg  water vapor pressure)  In addition, 180 categorical features were used, which are CORINE land cover classes at level 1 (5 classes), level 2 (15 classes) and level 3 (44 classes) and 116 soil types (SoilGrids)  Note that each location should belong to exactly 4 categories: one CORINE class at each of the three levels and a soil type 
Numerical features  Numerical features can be treated similarly to the tag occurrences, i e  we will assume that the value of a given numerical feature can be predicted from the location vectors using a linear mapping  In particular, for each numerical feature INLINEFORM0 we consider a vector INLINEFORM1 and a bias term INLINEFORM2 , and the following objective: INLINEFORM3
where we write INLINEFORM0 for set of all numerical features and INLINEFORM1 is the value of feature INLINEFORM2 for location INLINEFORM3 , after z-score normalization 
Categorical features  To take into account the categorical features, we impose the constraint that locations belonging to the same category should be close together in the space

ground_truth: Unanswerable

 Paraphrase Database (PPDB) ,  book corpus

Unanswerable

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. what are the existing approaches?
Context 3:  To formalize this, we represent each category type INLINEFORM0 as a vector INLINEFORM1 , and consider the following objective: INLINEFORM2
Evaluation Tasks
We will use the method from BIBREF7 as our main baseline  This will allow us to directly evaluate the effectiveness of embeddings for the considered problem, since we have used the same structured datasets and same tag weighting scheme  For this reason, we will also follow their evaluation methodology  In particular, we will consider three evaluation tasks:
Predicting the distribution of 100 species across Europe, using the European network of nature protected sites Natura 2000 dataset as ground truth  For each of these species, a binary classification problem is considered  The set of locations INLINEFORM0 is defined as the 26,425 distinct sites occurring in the dataset 
Predicting soil type, again each time treating the task as a binary classification problem, using the same set of locations INLINEFORM0 as in the species distribution experiments  For these experiments, none of the soil type features are used for generating the embeddings 
Predicting CORINE land cover classes at levels 1, 2 and level 3, each time treating the task as a binary classification problem, using the same set of locations INLINEFORM0 as in the species distribution experiments  For these experiments, none of the CORINE features are used for generating the embeddings 
In addition, we will also consider the following regression tasks:
Predicting 5 climate related features: the average precipitation, temperature, solar radiation, water vapor pressure, and wind speed  We again use the same set of locations INLINEFORM0 as for species distribution in this experiment  None of the climate features is used for constructing the embeddings for this experiment 
Predicting people's subjective opinions of landscape beauty in Britain, using the crowdsourced dataset from the ScenicOrNot website as ground truth  The set INLINEFORM0 is chosen as the set of locations of 191 605 rated locations from the ScenicOrNot dataset for which at least one georeferenced Flickr photo exists within a 1 km radius 
Experimental Setup
In all experiments, we use Support Vector Machines (SVMs) for classification problems and Support Vector Regression (SVR) for regression problems to make predictions from our representations of geographic locations  In both cases, we used the SVM INLINEFORM0 implementation BIBREF53   For each experiment, the set of locations INLINEFORM1 was split into two-thirds for training, one-sixth for testing, and one-sixth for tuning the parameters  All embedding models are learned with Adagrad using 30 iterations  The number of dimensions is chosen for each experiment from INLINEFORM2 based on the tuning data  For the parameters of our model in Equation EQREF7 , we considered values of INLINEFORM3 from {0 1, 0 01, 0 001, 0 0001} and values of INLINEFORM4 from {1, 10, 100, 1000, 10 000, 100 000}  To compute KL divergence, we need to determine a set of classes INLINEFORM5 for each experiment  For classification problems, we can simply consider the given categories, but for the regression problems we need to define such classes by discretizing the numerical values  For the scenicness experiments, we considered scores 3 and 7 as cut-off points, leading to three classes (i e  less than 3, between 3 and 7, and above 7)  Similarly, for each climate related features, we consider two cut-off values for discretization: 5 and 15 for average temperature, 50 and 100 for average precipitation, 10 000 and 17 000 for average solar radiation, 0 7 and 1 for average water vapor pressure, and 3 and 5 for wind speed  The smoothing parameter INLINEFORM6 was selected among INLINEFORM7 based on the tuning data  In all experiments where term selection is used, we select the top 100 000 tags  We fixed the radius INLINEFORM8 at 1km when counting the number of tag occurrences  Finally, we set the number of negative examples as 10 times the number of positive examples for each location, but with a cap at 1000 negative examples in each region for computational reasons  We tune all parameters with respect to the F1 score for the classification tasks, and Spearman INLINEFORM9 for the regression tasks 
Variants and Baseline Methods
We will refer to our model as EGEL (Embedding GEographic Locations), and will consider the following variants  EGEL-Tags only uses the information from the Flickr tags (i e  component INLINEFORM0 ), without using any negative examples and without feature selection  EGEL-Tags+NS is similar to EGEL-Tags but with the addition of negative examples  EGEL-KL(Tags+NS) additionally considers term selection  EGEL-All is our full method, i e  it additionally uses the structured information  We also consider the following baselines  BOW-Tags represents locations using a bag-of-words representation, using the same tag weighting as the embedding model  BOW-KL(Tags) uses the same representation but after term selection, using the same KL-based method as the embedding model  BOW-All combines the bag-of-words representation with the structured information, encoded as proposed in BIBREF7   GloVe uses the objective from the original GloVe model for learning location vectors, i e  this variant differs from EGEL-Tags in that instead of INLINEFORM1 we use the number of co-occurrences of tag INLINEFORM2 near location INLINEFORM3 , measured as INLINEFORM4  
Results and Discussion
We present our results for the binary classification tasks in Tables TABREF23 – TABREF24 in terms of average precision, average recall and macro average F1 score  The results of the regression tasks are reported in Tables TABREF25 and TABREF29 in terms of the mean absolute error between the predicted and actual scores, as well as the Spearman INLINEFORM0 correlation between the rankings induced by both sets of scores  It can be clearly seen from the results that our proposed method (EGEL-All) can effectively integrate Flickr tags with the available structured information  It outperforms the baselines for all the considered tasks  Furthermore, note that the PPMI-based weighting in EGEL-Tags consistently outperforms GloVe and that both the addition of negative examples and term selection lead to further improvements  The use of term selection leads to particularly substantial improvements for the regression problems 
While our experimental results confirm the usefulness of embeddings for predicting environmental features, this is only consistently the case for the variants that use both the tags and the structured datasets  In particular, comparing BOW-Tags with EGEL-Tags, we sometimes see that the former achieves the best results  While this might seem surprising, it is in accordance with the findings in BIBREF54 , BIBREF38 , among others, where it was also found that bag-of-words representations can sometimes lead to surprisingly effective baselines  Interestingly, we note that in all cases where EGEL-KL(Tags+NS) performs worse than BOW-Tags, we also find that BOW-KL(Tags) performs worse than BOW-Tags  This suggests that for these tasks there is a very large variation in the kind of tags that can inform the prediction model, possibly including e g  user-specific tags  Some of the information captured by such highly specific but rare tags is likely to be lost in the embedding 
To further analyze the difference in performance between BoW representations and embeddings, Figure TABREF29 compares the performance of the GloVe model with the bag-of-words model for predicting place scenicness, as a function of the number of tag occurrences at the considered locations  What is clearly noticeable in Figure TABREF29 is that GloVe performs better than the bag-of-words model for large corpora and worse for smaller corpora  This issue has been alleviated in our embedding method by the addition of negative examples 
 Conclusions
In this paper, we have proposed a model to learn geographic location embeddings using Flickr tags, numerical environmental features, and categorical information  The experimental results show that our model can integrate Flickr tags with structured information in a more effective way than existing methods, leading to substantial improvements over baseline methods on various prediction tasks about the natural environment 
Acknowledgments
Shelan Jeawak has been sponsored by HCED Iraq  Steven Schockaert has been supported by ERC Starting Grant 637277 

ground_truth: BOW-Tags, BOW-KL(Tags), BOW-All, GloVe