fabiochiu/medium-articles huggingface dataset:
Taiwanese Elections from an Environmental Viewpoint
By: Tony Yen
Tags: Taiwan, Climate Change, Renewable Energy, Energy Transition

Taiwanese Elections from an Environmental Viewpoint

Overall Summary of the Elections

China is still the dominant factor in the elections. Many progressive parties and voters have strategically endorsed the presidential candidate of the Democratic Progressive Party(DPP), forming an “united front against Chinese nationalism”.

Yet tensions exist between the allies because of topics related to environment, labor rights, etc. This resulted in the strong competition and fierce attacks between the political workers of DPP and other parties; due to the same pool of potential voters, their competition between each other might have been more intense than with the Chinese nationalist party and its allies.

The latest legally-allowed polls suggested that DPP led the presidential election. The situation in the parliament election is more uncertain.

Climate and Energy Policies

Most progressive parties endorse the overall idea of the energy transition, but call for long term climate and energy goals.

DPP keeps its energy transition goals to reach 20% renewables by 2025, and an additional 10 GW of offshore wind between 2025 to 2035. Other than that there exists no further official plans for renewable expansion. The greenhouse gas reduction goals adopted by the environmental protection agency in 2017 is still valid nominally, but it is by no means ambitious and there exists still no concrete road map to achieve it.

On the other side of the political spectrum, more vague promises are given. The Chinese nationalist candidate calls for a halt of the ongoing expansion of renewables until he considers them cheap enough to make economic sense. In his viewpoint that would be no earlier than 2025. However, he also promises that Taiwan could get 50% of low carbon electricity by 2035 (he intentionally dodges the question of how much renewables that would imply). As usual, his supporters don’t seem to bother with the paradox that he is planning to do less than the current government during his supposedly presidential term (2020–2024), while promising a similar mid term goal.

Although during the campaign, a large focus has been put on the remaining nuclear fleet and the unfinished NPP 4, the possibility of reviving the industry remains dim. The remaining two nuclear power plants are reaching their designed lifetime, and life extension would face large opposition from legal and safety viewpoints. NPP 4 has always been an engineer nightmare; Taipower estimated that it would take 7 years the least just to overcome the technical difficulties of completion, not to mention that new seismic faults have been found near the site recently. These older nuclear designs would also pose a threat to further renewable integration if they are to be kept on line.
Leofoo Village Theme Park
By: Connect Business With Technique
Tags: Taiwan, Zoo, Hsinchu, Travel

Leofoo Village Theme Park

I am gonna introduce Leofoo Village Theme Park, one of the theme parks in Hsinchu, Taiwan. This park is famous for its zoo and its incredible theme- designs.

If you are a big fan of roller coasters, drop towers or other exciting stuff, you can visit Leofoo Village for its 笑傲飛鷹, 大怒神, 老油井. I bet you will be bringing a lot of fun and exciting experience from those facilities.

Okay, here comes the most important part of this article, the zoo. What you can see in Leofoo Village? Well, pretty much, but the most special animal you can see here is the white tiger. I would say I am not a big fan of watching animals who are stuck in a tiny place, but the truth is these animals are not able to survive once we send them back to the wild. It's good news that white tigers here are still healthy and strong.

cute but poor white tiger

There is an area for those cute animals, you can bring your children here feeding the animals and even touch them. This is a place full of family love, couple love and sisterhood.

A hahaha

Monkeys are booming and under protection here in Taiwan. It is very easy to observe their group. You will find who is the king and their relationships within a group of monkeys.

Quite clear who is the king right

Welcome to this park to enjoy more and have more experience with animals, special performs and exciting facilities. The ticket information is here : http://www1.leofoo.com.tw/village/en/TicketPrice.aspx

Normally you have to pay full price, but don't forget you can purchase a discounted ticket from KKday or Klook. Here is one more thing to keep in mind that you can download its APP for more information and win a VIP-service including the priority of playing any facility.
Impeach Now — Part 2 — Diplomatic Ignorance
By: Gregory Boyce
Tags: Politics, Taiwan, China, Impeachment

I’m going to spell out why I believe Donald Trump needs to be impeached immediately.

Taiwan

This one I noticed due to part of my family that was living in Taiwan at the time.

Taiwan is an interesting country politically. When the communist takeover of China occurred, the existing government fled to the island of Taiwan. My understanding is that they exist there as the Republic of China after losing the mainland.

From the perspective of the People’s Republic of China (mainland) however, Taiwan is part of their country. Since Nixon’s visit to China, that has been US policy as well.

Nixon: …Principle one. There is one China, and Taiwan is a part of China. There will be no more statements made — if I can control our bureaucracy — to the effect that the status of Taiwan is undetermined. …Second, we have not and will not support any Taiwan independence movement. Third, we will, to the extent we are able, use our influence to discourage Japan from moving into Taiwan as our presence becomes less, and also discourage Japan from supporting a Taiwan independence movement… The fourth point is that we will support any peaceful resolution of the Taiwan issue that can be worked out. And related to that point, we will not support any military attempts by the Government on Taiwan to resort to a military return to the Mainland. Finally, we seek the normalization of relations with the People’s Republic. We know that the issue of Taiwan is a barrier to complete normalization, but within the framework I have previously described we seek normalization and we will work toward that goal and will try to achieve it.

There has been a tenuous peace where we trade with both countries but pretend that Taiwan isn’t self-ruled.

And then we elected a man who is dangerously unaware.

My uncle uprooted his family and moved back to the US, because the future of Taiwan is too uncertain.
Discovering the meaning of home in Taiwan’s pandemic
Tags: Covid 19, Hope, Pandemic, Personal, Taiwan

As the US exits the pandemic, Taiwan enters ours. In two weeks, domestic cases have shot from 0 to 554 towards an unforeseeable peak.

It’s deja vu in the least romanticized sense. When the same erupted in the States last year. I was first fleeing from Spring break in Nashville to St. Louis, then evacuating from St. Louis to Taiwan, all within three days.

Panic is the normal instinct. But, in America, these fears quickly mutated into a hateful division more terrifying than the coronavirus itself. As an Asian woman, I became not only a victim of the pandemic but also the perceived perpetrator.

“Americans will always see you as an outsider,” my parents’ preachings echoed in my head as I watched the rise in hate crimes from afar. Maybe that’s why it wasn’t so difficult to pack up my bags and leave so suddenly.

Once my friends all left St. Louis too, the city became drab and hollow. I wondered whether my roots in America were nourished by the country itself, or merely by the individuals who grounded me.

As the pandemic strikes Taiwan this time, I feel fear, but also trust and hope. I have faith in the Taiwanese people to emerge from this crisis. The streets are empty. Masks are fully accepted. Fake news is rare.

While criticisms still exist, there is even more social buzz urging for an attitude of solidarity instead. Though some friends have fled Taiwan, I still feel rooted in this island despite a diminishing community.

I can flee the pandemic again if I want to. This time, by flying to the US instead. But, there is a nagging feeling of responsibility. These are my people, my government, my home.

Over my country-hopping lifestyle these past few years, I’ve frequently returned to the question of “roots” and “home”. What makes a place home? What does it feel like to have roots? I wonder if herein lies the answer. This sense of belonging, even when shit hits the fan.

(Written on May 29th)
TTC & …
By: Lu Yeh
Tags: Trade And Tech, Transatlantic, Semiconductors, Taiwan, Ttc

今天要來發摟一個我很不熟的題目：跨大西洋科技貿易聯盟。拜登上任以來第一次出訪，去了歐洲。除了G7峰會，他也要去布魯塞爾，跟Von der Leyen一起宣布EU-U.S. Trade and Technology Council (TTC)。

去年年末，越接近美國選舉，各種重返跨大西洋合作的討論一時無兩；川普確定掰掰之後，布魯塞爾樂壞了，大西洋對面終於換回來一個穩定可預期的夥伴。在12月的時候，執委會（和外交事務委員會）就提出了A new EU-US agenda for global change。

著眼於歐盟與美國聯手對抗在全球範圍內，以科技大國之姿，快速崛起的中國。為什麼是聯盟呢？執委會提案建立一個強而有力的多邊機制，並支持理念相近夥伴國家（也就是願意遵守國際義務與普世價值的民主國家）加入。

這點其實有點乘著川普政府的勢頭，當時老川對中國窮追猛打，與一些中東歐國家共同發表雙邊聲明「不使用受外國政府控制廠商所提供的網路基礎設施」。不過儘管對抗中國，阻止中國成為數位科技領域的規則制定者，算是大西洋兩邊的共識，但在一些關鍵的問題上，像是增加對科技巨頭的管制，美國始終不願鬆口。畢竟規則一旦確立，那美國很難再為帶給美國政府龐大稅收的Amazon、Google、Facebook受到歐盟貿易調查而靠北。

The European Union and the United States will announce a wide-ranging partnership around technology and trade next week…

過了大半年， 雙方將在6月15日宣告TTC的成立與運作細節。雖然尚未公布任何文件，但根據去年底執委會的提案，促進科技創新、貿易規則設定等等，沒意外應該會是重點。

除此之外，整整一年來雙邊深受全球供應鏈斷鏈斷所苦，不斷強調重組全球供應鏈的必要這點（註：這甚至已是執委會被批評是浪費歐洲人血汗錢的2030數位羅盤計畫的一環）看來，科技基礎設施全球供應鏈的穩健必定是核心之一，但要怎麼個穩健法呢．怎樣才夠穩健呢．歐盟真要自己蓋半導體廠，還是他要仿效美國的作法，想辦法把台積電拉過去投資呢？

小英今年三月在哥本哈根論壇關於供應鏈合作是這麼說的：

Taiwan will continue to engage with the EU and other democratic partners to establish a more resilient supply of critical goods such as semiconductors and medical supplies,

然後他補充：

A BIA would not just help secure our supply chains; it would protect our mutual geopolitical and economic interests, and send a message about the partnerships and values on which our interests depend.
🇹🇼 🍺台灣第15任總統就職紀念的柑橘味，「台啤特釀 · 淬」啤酒（小七） Taiwan Beer Special Brewing for 15th Presidential Inauguration
By: Kyle Chu 微風捕手
Tags: Beer, Taiwanese, 啤酒, 中文, Taiwan

對印度淡色艾爾啤酒迷來說，台灣啤酒本是無緣，在台灣啤酒產品，找不到那種印度淡色艾爾啤酒獨有的柑橘味。

For IPA beer fans, Taiwan beer was not a familiar brewery. In Taiwan beer products, the unique citrus flavor of Indian light beer did not exist.

然而，「台啤特釀 · 淬」啤酒，似乎擁有與其他台啤產品不同的柑橘味。

However, the “Taiwan Beer Special Brewing for 15th Presidential Inauguration” beer seems to have a different citrus taste from other Taiwan beer products.

台灣啤酒官網上，說明這款啤酒用全麥及歐洲貴族啤酒花，選用果香型酵母，遵循比利時金色艾爾啤酒工法釀製而成，酒精度5.5%。

The official website of Taiwan Beer states that this beer is brewed with whole wheat and European noble hops, selected from fruit-flavored yeast, and brewed according to the Belgian golden Ayr beer method, with an alcohol content of 5.5%.

這款啤酒，泡沫不多，泡沫也不細緻， 但的確有明顯柑橘味，堪稱2020年台灣啤酒廠產品的首選。 This beer is not too foamy and the foam is not meticulous, but it does have a clear citrus flavor, which is the best choice for Taiwanese brewery products in 2020.

味道不差，反倒是名設計師聶永真設計的啤酒罐，銀底白字的設計，造成閱讀基本資料的困難，連酒精濃度的標示都要找好久

The taste is not bad, but rather designer Nie design of beer cans, silver and white on the design, resulting in difficulties in reading basic information.

不過，慶祝台灣第15任總統副總統就職，5/6起各通路都買得到的「台啤特釀 · 淬」啤酒，無疑是近年台啤少見的佳釀。

However, to celebrate the inauguration of the 15th President and Vice President of Taiwan, the “Taiwan Beer Special Brewing for 15th Presidential Inauguration” beer that has been available since 5/6 is undoubtedly a good brew of Taiwan Beer in recent years.
A brief summary of coronavirus in Taiwan.
By: Erin Lee
Tags: 2020, Taiwan, Coronavirus

Today, in 2020, there is a serious disease affecting the world called COVID-19. COVID-19 is a kind of coronavirus which comes from Wuhan China. This disease quickly spread across the world, and people who are infected with the disease get a fever and cough. Unfortunately, this disease can be spread from person to person.

Taiwan’s government noticed how serious this disease due to Taiwan’s experience with SARS in 2003. Immediately after the virus came to Taiwan, the government implemented policies including a prohibition on the export of mask and limit on mask purchases of 3 masks per week per person. Taiwan also limited entry from China and other countries that have serious outbreaks. Anyone who enters Taiwan has to stay at home at least 14 days and monitor their body temperature by themselves.

Taiwan’s government also gathered experts to develop a system to show people where they can buy masks, At the same time, other experts did their best to bolster mask production lines, making sure Taiwan had enough masks to protect people.

Some experts thought that Taiwan would suffer the most serious outbreak of this disease because of Taiwan’s location and connection with China. However, due to government policies and the great effort of medical teams, Taiwan has successfully controlled the disease and protected its people so far.
Talking Taiwan Episode with Author, Musician, and Journalist Joe Henley
By: Felicia Lin
Tags: Taipei, Punk, Authors, Novel, Taiwan

In this week’s Talking Taiwan episode, I spoke with Joe Henley, a Taipei-based author, freelance writer and musician about his latest book, Bu San Bu Si, and his first book, Sons of the Republic. In writing Bu San Bu Si, Joe clearly drew upon his multi-disciplinary background. The book after all is about a Taiwanese punk band and the struggles that the musicians face in reconciling their punk personas with Taiwan’s societal expectations. Joe , incidentally, has sung in punk rock and metal bands, and much of what he’s written in Bu San Bu Si was inspired by what he experienced in the Taipei punk and metal band scene.

We also talked about what motivates people with creative pursuits and how important money and fame might be in that pursuit. I asked him about his writing process and to offer advice to aspiring writers. Joe was unabashedly frank about what motivated him to write his first book, Sons of the Republic and talked about the subject of his next book.

Here’s a little preview of what we talked about in this podcast episode:

· Joe’s latest novel, Bu San Bu Si

· What the phrase “bu san bu si” means

· Joe’s writing process

· How Joe ended up in Taiwan

· How Saskatchewan (where Joe is from) compares to Taiwan

· How Joe adjusted from moving from Saskatoon to Taipei

· What motivated Joe to write his first novel, Sons of the Republic

· Joe’s career as a musician and freelance writer

· The topics of Joe’s future books

· Joe’s advice for aspiring authors

· Being a journalist vs. a novelist/author

· Joe’s two bands: Stench of Lust, and Revilement

Listen to my interview with Joe Henley by clicking HERE

This post originally appeared on www.TalkingTaiwan.com
Frozen II Serect
Tags: Time, Frozen, Taiwan, Blockchain, Disney

Parents say “when we return” Iduna told agree the truth is about “time travel”

This picture is about the time when F1 Elsa released her magic

White fonts are mostly fear, but there are also moments of happiness

The point is, they all shine

Some terror doesn’t shine, but it’s aggressive

Frozen II not over

PIC1 ∞PIC2

The end of the ice age, (time)

the river found but lost. (time)

Magic source. Elsa’s source?

I can Read ALL…

Awakens the Four spirit

Funnel represents time

River flow and reverse current

Day ∞ night Black ∞ white

Autumn leaves withered ∞ Grow up

Frozen 2 | Deleted Scene: “A Place of Our Own”

Why can Elsa recreate the memory! ? What does memory reappearance represent! ? Time reflux Because Elsa can control the time

Time Travel

Can save F1 Elsa and Anna.

and bring parents back.

She can go back to F1 to save herself It’s the third row from the top Black border on top represents time reversal The black border below represents the positive time flow To get rid of the countercurrent, Elsa’s positive current is two men and one woman. This is Elsa’s first time blocking the bow and arrow in F1

this time have Police voice

Is it the Northuldra? Or something?

Or someone?

And to keep Anna from freezing in an instant, to keep Hans from attacking her sister

And Elsa doesn’t feel it at that point.

Why Did Anna instantly freeze and avoid being attacked by Hans’ sword?

and avoid being attacked by Hans’ sword? but there are no shiny and have protective

And We know F1 Elsa unskilled ice magic Who protected Anna? Who else uses ice magic? If Elsa control time She is time travel What happened to F1 It would be very reasonable
Important Topics in Machine Learning You Need to Know
By: Sabina Pokhrel
Tags: Machine Learning, Supervised Learning, Unsupervised Learning, Artificial Intelligence, Evaluation Metric

Performance metrics for classification problems

Confusion matrix: It is one of the most intuitive and easiest metrics used for finding the correctness and accuracy of the model. It is used for classification problem where the output can be of two or more types of classes.

Confusion Matrix (Source)

True Positives (TP): are the cases when the actual class of the data point was 1 (True) and the predicted is also 1 (True).

True Negatives (TN): are the cases when the actual class of the data point was 0 (false) and the predicted is also 0 (False).

False Positives (FP): are the cases when the actual class of the data point was 0 (False) and the predicted is 1 (True). False is because the model has predicted incorrectly and positive because the class predicted was a positive one.

False Negatives (FN): are the cases when the actual class of the data point was 1 (True) and the predicted is 0 (False). False because the model has predicted incorrectly and negative because the class predicted was a negative one (0).

Accuracy: Accuracy in classification problems is the number of correct predictions made by the model over all the predictions made.

Accuracy in the confusion matrix (Source)

When to use accuracy: accuracy is a good measure when the target variable classes in the data are nearly balanced.

When not to use accuracy: accuracy should never be used as a measure when the target variable classes in the data are a majority of one class.

Precision (hits): Precision is a measure that tells us what proportion of predicted values as True is actually True.

Recall or sensitivity (misses): Recall is a measure that tells of what proportion of patients are actually true were predicted as being true by the model.

F1 score: Represents both precision and recall.

F1 Score (Source)

Receiver Operating Characteristic (ROC) curve: An ROC curve is a graph showing the performance of a classification model at all classification thresholds.

The curve plots two parameters:

True Positive Rate (Recall) False Positive Rate (Specificity)

ROC Curve (Source)

AUC (Area Under the ROC Curve): AUC measures the entire two-dimensional area underneath the entire ROC curve.

It provides an aggregate measure of the performance across all possible classification thresholds.
Two Ways to Learn Audio Embeddings
By: Edward Ma
Tags: Machine Learning, Data Science, Unsupervised Learning, Audio Engineering, Artificial Intelligence

Mel-frequency cepstral coefficients (MFCC), zero-crossing rate are some of classical feature for audio. It can be extracted via the library easily. However, it may not able to provide a high-quality signal or input for deep learning models nowadays.

Two teams of researchers propose a different way to learn audio embeddings but not leveraging those classical features. Chung and Glass (2018) proposes to learn word-based embeddings while Haque et al. (2019) suggests learning sentence-based embeddings.

Speech2Vec

Chung and Glass are inspired by word2vec to propose a different way to learn audio embeddings. word2vec leverages skip-gram or continuous bag-of-word (CBOW) to learn word embeddings. In short, word embeddings are learnt through neighbor words. If you do not familiar with it, you may visit the research paper or this story.

Speech2Vec adopts a similar concept which is learning acoustic embeddings via neighboring acoustic regions. The audio segment is split by word and feeding to the model to learn a fixed embedding of audio segment corresponding to a word.

Speech2Vec with Skip-gram and CBOW

Same as word2vec’s skip-gram approach, the target word predicts surrounding words with a pre-defined range k before target word and after target word. Speech2Vec’s continuous bag-of-words (CBOW) go in different ways. Rather than using target words to predict surrounding words, it uses surrounding words to predict target words.
Clustering method 3
By: Yuki Liu
Tags: Clustering, Machine Learning, Spectral Clustering, Clustering Algorithm, Unsupervised Learning

找出的特徵向量如同 PCA 等降維方法一般，但此對應著最佳劃分圖的方法。最終需要再將連續的特徵向量離散化，即為特徵的劃分，進而得到欲達成的分群。

－ 演算法 －

輸入：資料集 V ，以及分群數目 k

輸出：目標分群集合 Clusters

求出資料點的鄰接矩陣 A（Affinity matrix）。 對鄰接矩陣 A 的第 i 列加總作為對角矩陣 D 第（i, i）的元素值，並算出矩陣拉普拉斯矩陣 L 。 從矩陣 L 中找出對應前 l 個特徵值的特徵向量 x_1 , x_2 , … , x_l ，並建構 n x l 維的矩陣 X = [ x_1 | x_2 | … | x_l ]。 對矩陣 X 的每一列做正規化（R → [0 , 1]），得到 Y 矩陣。 將矩陣 Y 的每一列做為新資料點（即原資料點降維成 l 維的資料點），透過 K-means 或其他分群方法分成 k 類。 若 Y 的第 i 列被歸為第 j 類，則原資料點 s_i 即為第 j 類。

－ 算法實作 －

使用Sklearn.cluster.MeanShift套件：

import numpy as np

import matplotlib.pyplot as plt

from sklearn.cluster import SpectralClustering

from sklearn import datasets #create datasets

X,y = datasets.make_blobs(n_samples=50, centers=3, n_features=2, random_state= 20, cluster_std = 1.5) #parameter setting

n = 3 #number of clusters #Spectral Clustering method

model = SpectralClustering(n_clusters = n,

assign_labels = 'discretize')

model.fit(X)

labels = model.fit_predict(X) #results visualization

plt.figure()

plt.scatter(X[:,0], X[:,1], c = labels)

plt.axis('equal')

plt.title('Prediction') plt.show()

左圖為原始資料集，右圖為譜分群設定 n = 3 時的結果。

n = 2 時，譜分群就會將距離較近的兩群合為一類。

用於影像分割 …

import numpy as np

import matplotlib.pyplot as plt

from skimage.transform import rescale

from sklearn.cluster import SpectralClustering

import cv2 #load image

img = cv2.imread('AIA.png')

img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

img = rescale(img, 0.2)

rows, cols, chs= img.shape #convert image shape [rows, cols, 3] into [rows*cols, 3]

feature_img = np.reshape(img, [-1, 3]) #consider the data without density-dependent factor

uni_feature_img, idx = np.unique(feature_img, axis=0, return_inverse = True) #parameters setting

n = 15 #number of clusters #Spectral Clustering method

model = SpectralClustering(n_clusters = n, random_state = 40)

model.fit(uni_feature_img)

labels = model.fit_predict(uni_feature_img) #results visualization

fig = plt.figure(figsize = (20, 12))

ax = fig.add_subplot(121)

ax1 = fig.add_subplot(122)



ax.imshow(img)

ax1.imshow(np.reshape(labels[idx], [rows, cols])) plt.show()

此例僅考慮顏色的因素，並不考慮密度，因此同顏色的像素都將視為同一點。

左圖僅顯示被歸為背景藍色的像素；右圖則表示被歸為白色系的像素

－ Spectral Clustering算法總結－

優點：

只需要資料點間的相似矩陣，因此對於稀疏資料點的分群有顯著的效果。 此算法利用拉普拉斯矩陣降維，所以對高維度的資料集的分群計算複雜度相對低。

缺點：

若降維後的維度依舊很高，則會影響分群的速度也會導致分群的效果不好，因此選擇降維的 k 值尤為重要。 亦會因為相似矩陣的計算方式得到不同的分群效果。

－ 參考資料 －
Unsupervised Medical Image Segmentation Using Center of Mass Algorithm
Tags: Unsupervised Learning, Deep Learning, Medical Imaging

Image segmentation is an important step in many medical imaging applications. The general approach is to use supervised learning to train a deep learning model. However, deep learning models require large amounts of data, and manually annotating medical images is a challenging task for multiple reasons: it requires a medical expert, takes a lot of manual effort, and involves high economical costs.

These issues have led to a growing interest in unsupervised image segmentation. It doesn’t require labelled data, and can be either used as a direct substitute for supervised segmentation models, or as a starting point for manual annotation. Various approaches have been suggested for this, two of which are discussed here.

Local Center of Mass Approach

This approach was suggested by Aganj et al[1]. and is based on 1D local center of mass. It divides a 1D signal into different regions by using the information from the entire signal. It does so with a computational complexity of O(N). The idea is to define a weighting function between each observation in the 1D signal. The weighting is large if both points lie in the same regions, and small otherwise.Local center for the nth point is given by

Weighting between the mth and nth points is given by

where

Alpha and p are hyperparameters. Dn is positively related to the number of transitions/edges up to the nth point. Hence the difference between Dm and Dn will be low if the region between them has less transitions.

The formula for Cn ban be extended as

Using precomputation, the local centers for all points can be calculated in O(n) time complexity.

This approach is extended to 2D and 3D by considering multiple 1D orientations. This makes the approach suitable for detection of spatial segments. Since centerlines extend in one direction, the results obtained are not much suitable.

Author VaibhavJain (Kardiolabs.ai)
Day 1: Notes from Machine Learning mastery journey
By: Abhishek Sen
Tags: Machine Learning, Supervised Learning, Unsupervised Learning, Regression, Artificial Intelligence

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition Book: https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/

Photo by Braden Collum on Unsplash

No implement from scratch, only using frameworks

Learning: Intuitive => Project based => Less Theory => Even be read without picking up a laptop

What is machine learning ?

It is the science (and art) of programming computers so they can learn from data.

Types of ML Systems:

1- Based on human supervision required

- Supervised Learning:

Needs a lot of help. Ex Classifying mail as ham or spam. We individually label mail as spam or ham. Which machines consumes for learning. So we help a lot.

Usually its helpful for classification and regression(Ex: price of car).

Regression can be Categorical => Logistic regression

- Unsupervised learning:

No help from us. No labelling of data. Learn on your own.

This helps in categorizing => Ex. Have unlabelled blog visitors data => Categorize them=> Sci fi lovers => Comic book lovers etc

Dimensionality reduction => Simplify data without losing too much info => Two features Mileage and age can be correlated into one

Anomaly reduction => Reducing outliers => Train them with normal data=> if outlier comes they will flag => Same is Novelty detection => Just that Anomaly has some level of tolerance => To anomaly we can give some outliers => it will still perform well.

Association rule learning => People buying sauce and potato chips are also buying steaks => keep sauce, chips and steak counter closer

Next …
Customer-segmentation for differentiated targeting in marketing using clustering analysis
By: Debayan Mitra
Tags: Clustering, Machine Learning, Marketing Analytics, Customer Segmentation, Unsupervised Learning

Understanding the Dataset

We will use the famous shopping mall dataset (refer data here )— https://www.kaggle.com/vjchoudhary7/customer-segmentation-tutorial-in-python.

Let’s say you have data for 200 customers regarding their gender, age, income, spending score. Spending score is a proxy metric here used on a scale of 1–100. Higher spend score means customer has a higher affinity towards purchase of items and thereby higher revenue for the shopping mall. Our Goal is to best group these customers the best we can so that similar customers are grouped together and no 2 groups are similar.

source — kaggle

How would you do this ? You basically have 4 attributes of a customer and you need to decide the best segmentation you can achieve. Let’s say you had only the Gender attribute. Then, naturally you would group females and males into 2 different groups. or let’s say you had only the customer’s age — you would probably create age brackets (something like <18, 18–25, 25–35,35–50,>50 etc. )that would make business sense depending on the type of products your business sells to the customers. But what happens when you need to group customers basis multiple attributes (4 in this case). Let me explain you why the problem might get complicated. Let’s take a peek at the data and understand this.

Age vs Income — Male/Female

If a human see’s the above graph (Age vs Income graph by ‘Gender’), he would probably group basis some intuition on separation boundaries and some business context as follows :

clusters1

What about ‘spend score’ vs ‘age’ graph —

clusters2

Or may be Income and spend score like the graph below ?

clusters3

Or maybe something in the 3D space as it entails all the features in the dataset ?

3d scatter plot — age,income,spend

The dilemma in selecting the right segmentation method :

Marketing teams might choose cluster3 over cluster1 or cluster2 because they seem to be better separated. It also depends how many segments do we actually want to have ? More segments would mean more content creation for different groups. Too less segments can lead to over — generalization

This is just with 4 customer attributes. Real world problems would have 100s and 1000s of customer events and purchase data. The problem gets even more tougher then. Let us understand how we can approach this using machine learning algorithms. We will look at 3 common approaches (below) and compare results to the human labelled clusters :
Centroid Neural Network for Clustering with Numpy
By: La Tran
Tags: Clustering, Machine Learning, Centroid Neural Network, Clustering Algorithm, Unsupervised Learning

Centroid Neural Network for Clustering with Numpy

Photo by Alex Padurariu on Unsplash

Centroid neural network (CentNN) is an efficient and stable clustering algorithm that has been successfully applied to numerous problems. CentNN does not require a pre-determined learning coefficient but still yields competitive clustering results compared to K-means Clustering or Self-Organizing Map (SOM) whose results heavily depend on the initial parameters. For those who neither know about nor understand this great algorithm, you are welcome to read my explanation with visual examples here. And now, let’s make CentNN clear with several lines of code.

I would like to choose the 2-d data clustering problem as the explaining example in this post because it is very straightforward to understand and watch and as it is an efficient tool to explain clustering algorithms.

Firstly, let’s import several necessary libraries and generate a dataset with 10 centers.

Image by Author

Some necessary subroutines that are prepared for the implementation, everything is available at my github:

Again, I assume that all of you guys already understood this algorithm. Otherwise, I highly recommend that you read the theory before going ahead. You can check my explanation for this algorithm here.

Now, let’s start training the algorithm:

Step 1:

Find the centroid c for all data, then split c into 2 weights w1, w2 with a small Ɛ.

Here are the 2 weights:

[0.43965411 2.88785116]

[0.33965411 2.78785116]

Step 2:

Find the winner neuron for each x in X.

The result after epoch 0:

Image by Author

Step 3:

From epoch 1 to the end, keep finding the winner neuron for every single data and updating winner neurons and loser neurons until reaching the desired number of clusters.

The algorithm stops after 81 epochs for the example dataset:

GIF by Author

Final clustering result:

Image by Author

Final centroids:

[-9.66726188 6.62956078]

[-8.57296482 -8.31582399]

[ 9.30888233 -2.32652503]

[5.82090263 0.57715316]

[5.55404696 7.36294438]

[-1.32316256 7.86651609]

[-1.63557118 2.87845278]

[2.08983389 0.85729017]

[1.47692106 8.56662456]

[0.90838486 4.30719839]

In this post, I have introduced to all of you a tutorial on the implementation of the Centroid Neural Network (CentNN) algorithm using Numpy. You guys can find my implementation of CentNN here. If you feel it helps, please do not hesitate to give it a star. You are welcome to visit my Facebook page which is for sharing things regarding Machine Learning: Diving Into Machine Learning.

The next post in my series of CentNN will be the combination of vector quantization (block quantization) and CentNN in image compression application.

That’s all for today. Thanks for spending time!

References

[1] Centroid Neural Network: An Efficient and Stable Clustering Algorithm

[2] My implementation of CentNN with Numpy
Machine Learning Guide for Beginners
By: Mili Jolly
Tags: Machine Learning, Supervised Learning, Reinforcement Learning, Unsupervised Learning, Artificial Intelligence

Machine Learning is one of the fastest-growing fields in today’s era. This article will give you an overview of what it is? How it is different from Artificial Intelligence(AI)? And what are its different types?

Photo by Luke Peters on Unsplash

What is Machine Learning?

Just like humans learn from their past experiences and make informed decisions, Machines can also learn and similarly make predictions.

We can teach Machines to learn. Sounds fascinating!…right?

We provide them with certain datasets and using them the machine makes future predictions, Machine focuses on the use of data and algorithms to imitate the way that humans learn and gradually improves accuracy.

According to Arthur Samuel,

It is the subfield of computer science that “gives computers the ability to learn without being explicitly programmed”.

The machine recognizes the patterns formed within the data and makes new patterns depending on the fed data. Here, a specific code for a particular case is not required and the machine learns from its own experiences!
Explaining Supervised vs Unsupervised Machine learning methods through real-life examples.
Tags: Machine Learning, Supervised Learning, Data Science Training, Unsupervised Learning, Data Science

The illustration is taken from www.pexels.com

When we are in college, our teacher guides us and explains the information with examples. There is a sense of supervision involved when the teacher is explaining. The target information i.e. the answers are available.

A similar process occurs with supervised machine learning. Here we provide the training data to an algorithm with the goal of predicting outcomes.

However, as students, we need to work out what they will ask on exam day. An examination is like an ‘unsupervised’ learning task, you might have the same set of examples, but will not have the target information.

It’s you who has to find the answers or draw conclusions.

This is like unsupervised learning where the machine learns patterns from the data on its own.

Let me give you another example: Suppose you want to know more about the customers of your business.

You can have two questions in your mind:

Do our customers belong to different groups?

Here, no specific purpose or target is defined. It’s like, let’s explore and draw some conclusions. Such a problem statement is referred to as unsupervised.

Or your question can be:

2. Do some groups of customers have a higher chance of canceling our subscription?

Here, we are defining a specific target that will a customer leave or not. We are predicting the outcome. Such a problem statement is called supervised.

Classification, Regression and Causal Modelling are some of the supervised machine learning methods. Clustering is one of the unsupervised methods. But applying these methods to the data is the second step.

The first step is always to convert a business question into a machine learning problem.

Whether it will fall into supervised or unsupervised learning is an important beginning.
K-Means Clustering: Asian Cuisine Restaurant Startup Location Recommendation In Toronto
By: Zhehao Xu
Tags: Machine Learning, K Means, Data Visualization, Unsupervised Learning, Data Science

INTRODUCTION

It was sometimes confusing and frustrating when you were trying to start up a new restaurant business in Toronto but had not a clue where should your new restaurant locate at that could maximize your possibility to success and make a profit from it. In this capstone project, a closer analysis of the best locations for starting up a new Asian cuisine restaurant was conducted using data from multiple sources such as Foursquare, Kaggle. Insights would be provided to those who would like to start an Asian restaurant business in Toronto as the final delivery of this project.

DATA

The data this project needed was mainly focusing on factors that could influence the success of a restaurant including:

Competition

Area Average Income

Target Customers

Safety

Each above feature would be explained here why they were chosen to evaluate a place if it was an ideal candidate. The ‘Safety’ here was considering if the location had a low crime rate or not as an unsafe area may have a greater chance of getting robbed and this section of data was acquired from the robbery dataset on Toronto public safety data portal website. ‘Area Average Income’ was the average income in each neighborhood as it indicated how much customers could afford as a large portion of target customers would live nearby in the same area. ‘Competition’ was a measure of how many similar Asian cuisine restaurants were in that neighborhood that may compete with. ‘Target Customer’ was the group of customers that the Asian restaurant aimed to attract. It was important that you open the restaurant in a place with a large number of potential customers. In this case, Asians would be a good targeting group as people tended to have meals that they were familiar with.

METHODOLOGY

In order to reach the goal of providing valuable recommendations for new Asian cuisine restaurants, A formal comprehensive dataset must be constructed to drive insights which required multiple data sources as each one of them contributed single or multiple features that may help in evaluating a location following the above disciplines discussed in the data section.

A) Data Collecting & Cleaning

Neighborhood

The initial step of constructing the dataset was to confirm the primary key for it which in this case would be names of each neighborhood in Toronto. By searching on the Internet, there were in total 140 neighborhoods in the city of Toronto. The original plan was to use geopy to find the latitudes and longitudes for all those neighborhoods. However, after discovering some neighborhoods could not be located this way, an alternative solution was to import the neighborhoods dataset from the city of Toronto open data website which contained all 140 neighborhoods along with their latitudes and longitudes. Since the project did not interest in neighborhoods’ geometry data, only name, latitude, longitude for each neighborhood remained.
Unsupervised Representation Learning on Distributed Regions in the Human Brain (Part-IV)
By: Can Kocagil
Tags: Machine Learning, Neuroscience, Visualization, Unsupervised Learning, Dimensionality Reduction

Unsupervised & Manifold Learning in Human Brain

Functional MRI data are very high-dimensional if one considers all the voxels or surface coordinates acquired with standard imaging parameters. As in our dataset, with the structure of 4D time-series image data, we have a curve of dimensionality problem. Hence, dimension reduction and manifold learning algorithms can reduce the dimensionality of fMRI space by preserving geodesic relations in the lower space representations. We performed PCA, LDA, ICA, NNMF, and MDS as dimension reduction algorithms. Besides, t- SNE, UMAP, ISOMAP, LLE, and Spectral Embedding are performed to generate lower-dimensional manifolds of the fMRI space. Let’s start discovering the intersection of unsupervised learning and the human brain. I performed many unsupervised learning algorithms as it further helps to understand geodesic information underlying the human brain and gives prior information on whether the neural activities in distributed regions are decodable or not.

Let’s install and import all necessary packages. Please refer to previous articles (part-I) for the dataset understanding. Note that this article is not for answering the following questions (which are previously answered)

How fMRI data is structured? (part-I)

Why spatio-temporal masking is performed on the human brain? (Part-II)

Here are the pip commands for installing everything we need.

We installed everything (even more) we need. Let’s import them as follows.

We are ready to go! Let’s fetch the Haxby dataset.

Next thing is to prepare, mask, standardize and convert fMRI data to NumPy matrix form as follows.

Preprocessing part is done. So, we can move to the actual processes we want to perform. In this code snippet, “masks” are the masked regions in the human brain that we want to perform cognitive tasks on it and categories are just their labels. We do not use whole voxels for representation learning purposes. Our main idea is to extract and visualize the latent variables in the human brain where the distributed and overlapping patterns of neural activity are happening.

Finally, we can start the actual business as follows. In the following figures, different colors represent different categories. (i.e., blues belong to class 1, oranges belong to class 2 etc.)

To properly visualize the latent space, I utilized the plotly python package. The following code performs 2D and 3D visualization in an interactive fashion.

Dimension Reduction: PCA

PCA is a linear unsupervised dimension reduction algorithm and it computes principal vectors to change the basis of the representation [26]. PCA is a used algorithm in a broad range of topics from image compression to decorrelation of texts. Here, we performed PCA on RoI’s of subject 5 and visualized it as follows.
V measure: an homogeneous and complete clustering
By: Filippo Valle
Tags: Clustering, Metrics, Unsupervised Learning, Mutual Information, Entropy

Once the clustering is run one has to evaluate qualitatively the goodness of the output. First of all, to verify that groups are actually meaningful, and moreover having a score can be the proxy to evaluate different models and choose the best one.

There are many different scores out there, each one with its pros and cons.

One of these measures is the so-called V-measure (or Normalised Mutual Information) score. One of the advantages of this measure is that it can be factorised into two easily to visualise metrics.

Normalised Mutual Information

This score can be interpretated as an average of other two measures: homogeneity and completeness.

Homogeneity

Homogeneity measures how much the sample in a cluster are similar. It is defined using the Shannon’s entropy.

homogeneity. Image by Author.

Given H(C|K) formula.

It is easy to understand why this is the case. If one looks at the term H(C|K) it contains n𝒸ₖ / nₖ which represents the ratio between the number of samples labelled c in cluster k and the total number of samples in cluster k.

When all samples in cluster k have the same label c, the homogeneity equals 1.

Note that there is a sum over c and k, it doesn’t matter which is the cluster that contains a particular label, it is sufficient that there is at least one. This is useful when running unsupervised methods whose output has nothing to do with the labels.

Here an example of homogeneous clustering. All three clusters contains only one color (feature), in other words all the samples in the same cluster share their label.

Example of homogeneous clustering. Image by Author.

Anyway this situation is not optimal since orange samples are split into different clusters . In this example, a feature is not complete.
Visualizing large datasets with H2O
By: Parul Pandey
Tags: Aggregator, Outliers, Data Visualization, Unsupervised Learning, H2oai

Visualizing Large Data

Let’s now see how we utilize H2O to visualize a large dataset.

1️⃣. Initialize an H2O cluster

H2O has an R and Python interface along with a web GUI called Flow. In this article, however, we’ll use the Python interface of H2O. Every new session begins by initializing a connection between the python client and the H2O cluster.

A cluster is a group of H2O nodes that work together; when a job is submitted to a cluster, all the nodes in the cluster work on a job portion.

To check if everything is in place, open your Jupyter Notebooks and type in the following:

% matplotlib inline

import matplotlib.pyplot as plt

plt.style.use('fivethirtyeight') import h2o

h2o.init()

This is a local H2O cluster. On executing the cell, some information will be printed on the screen in a tabular format displaying, amongst other things, the number of nodes, total memory, Python version, etc.

Running h2o.init() (in Python) | Image by user

2️⃣. A first look at data 👀

Next, we will import the airline dataset and perform a quick exploration.

path = "https://s3.amazonaws.com/h2o-public-test-data/bigdata/laptop/airlines_all.05p.csv" airlines = h2o.import_file(path=path)

Let’s check the first ten rows of the dataset.

airlines.head()

First ten rows of the dataframe | Image by Author

What we have above is called an H2O Frame. It is similar to Pandas’ dataframe or R’s data.frame . One of the critical distinctions is that the

data is generally not held in memory. Instead, it is located on a (possibly remote) H2O cluster, and thus H2OFrame represents a mere handle to that data.

We can also take a look at a quick statistical summary of our dataset with the .describe() command as shown below

airlines.describe

Statistical summary of the dataframe | Image by Author

3️⃣. Visualising data

The dataset has around five million rows and thirty-one columns. Let’s now quickly plot a histogram of the Year column to see if there is a pattern in the data.

%matplotlib inline

airlines["Year"].hist()

Frequency of flights | Image by Author

Interestingly, the histogram shows that we do not have data for some years. This is the power that EDA brings to analysis. It helps to quickly pinpoint anomalies in datasets like missing values, outliers, etc.

Next, let’s plot the Departure versus the Arrival time to see if there is a relation between the two. This time we’ll plot a scatterplot so that we can see individual points.

# Convert H2O Frame into Pandas dataframe for plotting with matplotlib airlines_pd = airlines.as_data_frame(use_pandas = True) plt.scatter(airlines_pd.DepTime, airlines_pd.ArrTime)

plt.xlabel("Departure Time")

plt.ylabel("Arrival Time")



Scatterplot of Departure versus the Arrival time | Image by Author

As stated above, we do get a scatterplot, but the individual points are hardly discernable. A lot of overlapping points make it difficult to see a general trend in the data.
Succeeding Lewa!. Using Machine Learning to find…
By: Raj Sangani
Tags: Clustering, Machine Learning, Visualization, Unsupervised Learning, Data Science

The Dataset and K-Means :

I have used the Fifa 21 dataset from Kaggle for this approach. The link contains datasets from Fifa 15 up to 21. Now coming to K- Means Clustering.

Since the problem requires finding similar traits to those of Lewandowski’s I decided to use an unsupervised technique in clustering to find the similarities.

K-Means Clustering is a technique in which given the number of clusters N, data can be divided into these N different clusters based on the features of each data entry. The features are numeric and the clusters are formed iteratively. Data within each cluster is similar to other data within the same cluster. This similarity can be decided based on different metrics two of them being the Euclidean Distance and the Manhattan Distance. To know more about K-Means please refer to this link.

The features that I chose for clustering the players were based on physical and technical attributes and were more focused on a striker’s attributes. So don’t expect Messi to show up in the same cluster as Lewandowski since they are miles apart physically!