Introduction
Since Satoshi Nakamoto published the article "Bitcoin: A Peer-to-Peer Electronic Cash System" in 2008 BIBREF0 , and after the official launch of Bitcoin in 2009, technologies such as blockchain and cryptocurrency have attracted attention from academia and industry. At present, the technologies have been applied to many fields such as medical science, economics, Internet of Things BIBREF1 . Since the launch of Ethereum (Next Generation Encryption Platform) BIBREF2 with smart contract function proposed by Vitalik Buterin in 2015, lots of attention has been obtained on its dedicated cryptocurrency Ether, smart contract, blockchain and its decentralized Ethereum Virtual Machine (EVM). The main reason is that its design method provides developers with the ability to develop Decentralized apps (Dapps), and thus obtain wider applications. A new application paradigm opens the door to many possibilities and opportunities.
Initial Coin Offerings (ICOs) is a financing method for the blockchain industry. As an example of financial innovation, ICO provides rapid access to capital for new ventures but suffers from drawbacks relating to non-regulation, considerable risk, and non-accountability. According to a report prepared by Satis Group Crypto Research, around 81% of the total number of ICOs launched since 2017 have turned out to be scams BIBREF3 . Also, according to the inquiry reveals of the University of Pennsylvania that many ICO failed even to promise that they would protect investors against insider self-dealing.
Google, Facebook, and Twitter have announced that they will ban advertising of cryptocurrencies, ICO etc. in the future. Fraudulent pyramid selling of virtual currency happens frequently in China. The People’s Bank of China has banned the provision of services for virtual currency transactions and ICO activities.
The incredibly huge amount of ICO projects make it difficult for people to recognize its risks. In order to find items of interest, people usually query the social network using the opinion of those items, and then view or further purchase them. In reality, the opinion of social network platforms is the major entrance for users. People are inclined to view or buy the items that have been purchased by many other people, and/or have high review scores.
Sentiment analysis is contextual mining of text which identifies and extracts subjective information in the source material and helping a business to understand the social sentiment of their brand, product or service while monitoring online conversations. To resolve the risks and fraud problems, it is important not only to analyze the social-network opinion, but also to scan the smart contract vulnerability detection.
Our Proposed Methodology
We proposed two methodologies which integrate Long Short Term Memory network (LSTM) and Convolutional Neural Network (CNN) into sentiment analysis model, one outputs the softmax probability over two kinds of emotions, positive and negative. The other model outputs the tanh sentiment score of input text ranged [-1, 1], -1 represents the negative emotion and vice versa. Fig. 1 shows our system flow-chart and Fig. 2 is our system architecture. Detail descriptions are explained below.
Tokenizing and Word Embedding. The raw input text can be noisy. They contain specific words which could affect the model training process. To clean these input text, we use the tokenizer from Stanford NLP BIBREF4 to remove some unnecessary tokens such as username, hashtag and URL. Word embedding is a distributed representation of a word, which is suitable for the input of neural networks. In this work, we choose the word embedding size d = 100, and use the 100-dimensional GloVe word embeddings pre-trained on the 27B Twitter data to initialize the word embeddings.
Model Architecture. The architecture of our models are shown in Figure 3 , both based on the combination of LSTM and CNN.
Training. We use a max input sequence length of 64 during training and testing. For LSTM layer, the number of hidden units is 64, and all hidden layer size is 128. All trainable parameters have randomly initialized. Our model can be trained by minimizing the objective functions, we use $\ell _{2}$ function for tanh model and cross-entropy for softmax model. For optimization, we use Adam BIBREF7 with the two momentum parameters set to 0.9 and 0.999 respectively. The initial learning rate was set to 1E-3, and the batch size was 2048. Fig. 4 is our experiment result.
Ranking. The design of the model can guarantee the accuracy of the sentiment score of each text. However, when calculating the project score, problems occur when projects have the same score. For example, when the score of item A and item B are the same, while item A has 1000 texts, and item B has only 1 text. If it is only a simple calculation of the weighted score, it does not reflect the difference in the difference between the amount of data between Project A and B. So we designed a score-based scoring algorithm (as shown in the equation 16 ).
Datasets and Experimental Results
We train our models on Sentiment140 and Amazon product reviews. Both of these datasets concentrates on sentiment represented by a short text. Summary description of other datasets for validation are also as below:
We have provided baseline results for the accuracy of other models against datasets (as shown in Table 1 ) For training the softmax model, we divide the text sentiment to two kinds of emotion, positive and negative. And for training the tanh model, we convert the positive and negative emotion to [-1.0, 1.0] continuous sentiment score, while 1.0 means positive and vice versa. We also test our model on various models and calculate metrics such as accuracy, precision and recall and show the results are in Table 2 . Table 3 , Table 4 , Table 5 , Table 6 and Table 7 . Table 8 are more detail information with precisions and recall of our models against other datasets.
From the above experimental results, it can be found that the dataset from IMDB and YELP do not have neutral problem, so it can be solved with softmax. However, when we compare SST with SWN, we found many neutral sentences inside, which causes Softmax achieved poor results. Then we tried to use tanh, it got better results, but because of the data set, the current effect is still lower than other papers, but it is already better than using softmax! Therefore, we also tried to introduce the concept of emotional continuity in this work, we express the emotion as the output space of tanh function [-1, 1], and -1 represents the most negative emotion, +1 Representing the most positive emotions, 0 means neutral sentences without any emotions. Compared to other dichotomies or triads, we think this is more intuitive. We have listed the scores using the Yelp dataset as an example and divided them into positive (0.33, 1), neutral [-0.33, 0.33] and negative [-1, 0.33) emotions and each three representative sentences were presented in Table 9 (the score is rounded to 5 decimal places).
In order to validate the concept of emotional continuity, we re-deploy the rating of the product of cell phones and accessories reviews (Amazon product reviews) from five sentiment scores (5: very positive, 4: positive, 3: neutral, 2: negative, 1: very negative) to three sentiment scores ((1, 0.33): very positive and positive, [0.33, -0.33]: neutral, -0.33, -1]: negative and very negative) and re-train our model. We test our tanh and softmax model on other Amazon product reviews datasets (musical instruments, home and kitchen and toys and games etc... ) and calculate metrics such as accuracy of neutral reviews and all reviews. With this experimental results, we found tanh model got better results and can be solved neutral problem. We show the results are in Table 10 .
Our system run on a 64-bit Ubuntu 14.04, and hardware setting are 128 GB DDR4 2400 RAM and Intel(R) Xeon(R) E5-2620 v4 CPU, NVIDIA TITAN V, TITAN XP, and GTX 1080 GPUs; the software setting is the nvidia-docker tensorflow:18.04-py3 on NVIDIA cloud. More specifically, our training models run on a single GPU (NVIDIA TITAN V), with roughly a runtime of 5 minutes per epoch. We run all the models up to 300 epochs and select the model which has the best accuracy on the testing set.
Related Work
The sentiment of a sentence can be inferred with subjectivity classification and polarity classification, where the former classifies whether a sentence is subjective or objective and the latter decides whether a subjective sentence expresses a negative or positive sentiment. In existing deep learning models, sentence sentiment classification is usually formulated as a joint three-way classification problem, namely, to predict a sentence as positive, neutral, and negative. Wang et al. proposed a regional CNN-LSTM model, which consists of two parts: regional CNN and LSTM, to predict the valence arousal ratings of text BIBREF11 . Wang et al. described a joint CNN and RNN architecture for sentiment classification of short texts, which takes advantage of the coarse-grained local features generated by CNN and long-distance dependencies learned via RNN BIBREF12 . Guggilla et al. presented a LSTM and CNN-based deep neural network model, which utilizes word2vec and linguistic embeddings for claim classification (classifying sentences to be factual or feeling) BIBREF13 . Kim also proposed to use CNN for sentence-level sentiment classification and experimented with several variants, namely CNN-rand (where word embeddings are randomly initialized), CNN-static (where word embeddings are pre-trained and fixed), CNN-non-static (where word embeddings are pre-trained and fine-tuned) and CNN-multichannel (where multiple sets of word embeddings are used) BIBREF9 .
Besides, about using machine learning to solve fraud problems, Fabrizio Carcillo et. al proposed to reduce credit card fraud by using machine learning techniques to enhance traditional active learning strategies BIBREF14 . Shuhao Wang et. al observes users’ time series data in the entire browsing sequence and then uses in-depth learning methods to model the detection of e-commerce fraud BIBREF15 . As blockchain and virtual currencies prevail, fraudulent transactions cannot be ignored. Toyoda et al. identify characteristics of fraudulent Bitcoin addresses by extracting features from transactions through analysis of transaction patterns BIBREF16 ; Bian et. al predict the quality of ICO projects through different kinds of information such as white papers, founding teams, GitHub libraries, and websites BIBREF17 .
Conclusion
We developed a LSTM + CNN system for detecting the sentiment analysis of social-network opinion in this paper. Compared with the baseline approach, our system obtains good results. Our goal is to optimize the amount of parameters, network structure, and release automated detection tools, public RESTful API and chatbot. The future work is to reduce the complex task and train for higher performance in confronting the sentiment, in order to improve sentiment analysis. We also built a "help us" website to label more datasets is shown in Fig. 5 . The experiment material and research results are shown on the website if there are any updates.
More importantly, we collected the users’ cryptocurrency comments from the social network (as shown in Table 11 ) and deploy our sentiment analysis on RatingToken and Coin Master (Android application of Cheetah Mobile Blockchain Security Center). Our proposed methodology can effectively provide detail information to resolve risks of being fake and fraud problems. Fig. 6 is the screenshot of our sentiment analysis production website.
Acknowledgement
This work would not have been possible without the valuable dataset offered by Cheetah Mobile. Special thanks to RatingToken and Coin Master.

Question: Was the introduced LSTM+CNN model trained on annotated data in a supervised fashion?
Answer: Yes