fabiochiu/medium-articles huggingface dataset:
What makes an accent appealing?
By: Loredane Nolent
Tags: Accent Reduction, Komai, Polyglot, Language Learning, Accents

Accents are tricky things. When we start to learn a language, we need to decide which accents we want to follow. For example, Spanish from Spain or Latin America? French from France or Canada? There are no good or bad accents in the world. The perception of accents largely depends on our social and cultural associations.

What is an accent?

An accent is how someone shapes her mouth, moves her lips and flaps her tongue while forming words. This is basically how a person speaks.

Everyone has an accent. Your accent gives information about your age, social status, ethnicity, and whether or not the language is the speaker’s native tongue. It secretly reveals your identity.

Babies are quick language learners. Infants as young as 5 months can tell the difference of accents. They tend to respond to people speaking in their native accents because subconsciously, they want to seek to join a community of like-speaking people who share similar cultural values and will be able to communicate with others.

This accents are hints to find your own tribe.

How do we perceive if an accent is appealing?

This is largely subjective. People use accents to categorize people based on a general set of attitudes and behaviors that they think the accents represent those speakers.

According to a survey, many appealing accents are western languages because they associate with wealth and a history of colonization.

French and Italian are the most attractive language. This is not difficult to understand because these two languages are considered as romantic and love languages.

Contrastingly, German is not considered as a sexy language because people associate it with machinery and discipline,which German are famous for.

For a lot of American ladies, British accent is the hottest because it is associated with posh status and diplomacy. So when someone speaks the Queen’s English, it portrays an image of educated scholars and properness.

Thus, accent hierarchy does exist. When people favour certain accents and decide what’s appealing and what’s not. Those who are not in the group would be considered as inappropriate and become less -preferable as a language. For example, Russian, Dutch and Turkish.

Choose what suits you most

Although not all accents are weighted equally, it is not the main concern in learning a language. You should choose an accent that suits well in your situation.

For example, if you intended to study or work in Spain, then learning Spain Spanish will be a wiser choice.

Or if you are interested in French culture, then French accent is your option.

Don’t let certain accents stop you from learning a language.
Hacks for learning a new language
Tags: Productivity Tips, Learning Hacks, Skillup, Language Learning, Tts

This story has been published on the elocance blog! Check it out!

Learning a new language can be a confidence booster and if you’ve found us, it’s because you’ve either started or are planning to start learning one. Whether it’s a second, third, or fourth language, learning a new skill can be daunting. Finding tricks and hacks for making the process easier and faster, can be helpful.

1. Micro-dose — Studying for even 10–20 minutes daily has been proven to help you integrate a new language in your life. However, studying or learning for long periods of time like you’re binge watching Netflix, doesn’t really cut it.

2. Study before bedtime — Studying before sleeping at night helps you unwind while training your brain to accept information. Of course, if you start dreaming in another language then you may not need these tips afterall.

3. Gamify — Learning something new doesn’t have to be boring or stressful when it feels like a game. It is easier to succeed using an interactive method to learn languages in an enjoyable and engaging way. For example, apps like DuoLingo turn boring lesson plans and make them feel like a video game, boosting your motivation to master a language.

4. Change it up — Trying to employ multiple tools and methods can help you stay engaged. For example, watching YouTube videos, listening to the news in another language and reading children’s books in that language, can all help you get ahead faster. Text-to-speech apps like elocance help by reading written text out loud to you, helping you learn how to pronounce phrases and words properly.

5. Get out there — In addition to learning through media, connecting with natives is an important way to learn and practice speaking. Immersing yourself in a new culture is also one of the most enjoyable ways to get involved. For example, taking a cooking class or joining a Meetup (meetup.com) can be an enjoyable way to meet new people who have similar interests. Another great way to speak to a native is virtually. For example, the app hellotalk connects people from various countries to allows them to chat through video, text and more!

When learning a new language, letting go of fear and diving in are the most important steps. The rest of it should be fun!

Happy learning!

Team elocance

elocance is your audio superpower. Build productive habits and save time by listening. Save articles and PDFs in the app, turn them to audio and listen on the go.
the gods are restless
By: Sharon Hope Fabriz
Tags: Writing Life, Language, Creativity, Curiosity, Language Learning

painting (unidentified artist) from Swedish great-grandparents on my mother’s paternal side

The gods are restless. I typed too fast without reading the words and there it is. The gods are restless. I meant to type dogs.

I’m a fan of the fact that god is dog spelled backwards. I wonder at the tour de force that made that true. The evolution of language has a wisdom all its own.

I’ve been studying Swedish on Duolingo now for a year and a half. The hobby has been one of my COVID Stay Sharp and Challenge Yourself tests of will. My current playing streak is 180 days. I lost my even longer run when a migraine put me to bed for two days, and I couldn’t think straight in English much less Swedish. My spirit dragged when I saw my proof of engagement plummet to zero. So this is what it feels like to die in a video game, I thought. Still, I hold more knowledge than before.

Jag alskar dig. I love you. God morgon. Good morning. Kvinnor scriver. The women write. Hundarna springer. The dogs run.

Swedish is the language of my foremothers and fathers. My mother can still invoke the lilts of tone that bring Swedish expressions to life. She taught me what she thought I needed to know: tack sa mycket — thank you very much. I remember my grandmother reading letters to me in her native language from relatives in Sweden. Who were they? Where did they live? I wish I had paid more attention there on her front porch on 42nd Street South during our family’s summer visits to Minneapolis.

I imagine a family who speaks no English, who banters and cuddles and coos in Svenska not Engelska. Who looks upon a landscape that holds a variation of human memory unknown to me. A family whose sky gives more light in summer and more dark in winter. A family who lives under the aurora borealis and understands snow and ice like I never have. A family who fishes and farms and relies on the turning of seasons more than I do. What of their memory’s DNA lives in me? What of their innate wisdom travels in my bones?

A few days ago, I wrote a song in Swedish. The tune is an unsettled ballad, notes still in flux. The lyrics fill the restless space in my heart between me and the ancestors I have yet to know.
You won’t learn a language through apps alone
By: Timothy Dale
Tags: Duolingo, Language, Learning, Language Learning, Articles

I come from a generation that learned languages the old way, through books and television. Today however, many people are learning languages through various apps. Most apps are trying to approach languages in a fun way and they try to gamify the process, but that leads to failure. The points and achievements that are there to help you, soon turn to be a burden.

You are human, you will make mistakes, you will have bad days, you will fall down on the scoreboard. The traditional way of learning with a book does not have those problems. You will be more focused on acquiring new words for your vocabulary. Each day you learn will be a success, you will learn something new and the lack of the game system will free you from the pursuit of points.

One more important thing about learning without apps is that you can focus on the areas that interest you. Have some background in the targeted language? You can skip the boring parts you already know. Now I am not saying language apps are usless but people have to be aware they are just one tool in the sea of tools that are available to you.

Ask anyone who learned a language, they will all tell you it was a combination of books, apps and television. To keep yourself motivated you must love certain aspect of the targeted language or its culture, even games get boring. Remember key to learning a language is building your vocabulary, the rest will follow.
How To Say Beautiful In Serbian
By: Ling Learn Languages
Tags: Beautiful, Serbian, Language, Language Learning, Vocabulary

If you want to know how to say beautiful in Serbian, you probably want to flirt with someone and definitely should read this article. We hope this will help you to understand Serbian better.

Some Information About Serbians

People from Serbia are really polite and easy-going. They are always ready to help if anyone is in trouble. Serbians are very cheerful and joyful people that say “Živeli” (cheers in Serbian) all the time. So, that’s the reason why it’s not difficult for them to make new friends. In fact, I dare to say that while they are from Serbia, their hearts belong to the whole world. Trust me on this one, I am from Serbia and know what I am talking about.

If you start to talk to some person from Serbia, you should avoid using their full name unless you really have to. Serbians use full names with the people they don’t know or when they want to highlight that there’s something really serious to talk about. In most of the cases, this is when something bad has been done by someone and they would like to let that person know about it and that’s why they use their full name.

Why Should You Learn How To Flirt In Serbian?

If you really like someone, I think it can be very useful to learn some romantic words and phrases in the native language of that person. I would appreciate that kind of gesture and would be impressed by that person even more.

This can give you a serious chance of making them fall in love with you. If you like someone who is from Serbia, or from any part of what used to be Yugoslavia, you should try to learn them to say beautiful in Serbian, along with some other romantic phrases.

The main purpose of this article is to convince you to start learning something polite and nice. Trust me, there is nothing better than love phrases because I am sick and tired of the fact that people often tend to learn bad words first.

So let’s get started with learning how to say beautiful in Serbian:

Леп си. (Lep si.) — male

Lepa si. (Lepa si.) — female

Other Ways To Say Beautiful In Serbian

Compliments In Serbian

Final Thoughts

When you like someone, you have to say it. You should say beautiful in Serbian or use some of these other compliments to give your opinion about their personal qualities or the physical aspects of that person who is so special to you.

However, you should only tell that person what you really think and don’t use them too often! Otherwise, people might think you want a favor or something.

Are you interested in learning more words and phrases in the Serbian language like how are you in Serbian, for example? For extra reinforcement in the Serbian language, be sure to check out the Ling App. This great language training app features fun mini-games and quizzes that can motivate you to practice and explore more about the language you want to learn day by day.
Wanna learn Chinese? Ride the Shanghai Metro.
By: Jeremy Cadiz
Tags: Chinese, Metro, Language Learning, Language Acquisition, Shanghai

Photo by Sean Lim on Unsplash

For most expats who live in China, learning Chinese can be a daunting undertaking: all those characters, strange sounds, and those tones (!!) make you wonder if it’s even worthwhile. Well, to any expat living in Shanghai, I’ve found a way to learn Chinese in a very simple, yet passive way: the Shanghai Metro.

Impossible you say? 不是！(no way!). It has worked for me, so of course it can and will work for you! Let’s face it: most of us are way too busy or too broke to take a formal Chinese class, but on the flipside pretty much everyone who lives in Shanghai uses the Metro as their primary mode of getting from Point A to B. Why not make most the most of your time in the subway and learn some Chinese!

Before we begin, a few disclaimers:

While you won’t become fluent overnight, you will learn quite a few characters and how they are pronounced.

It will be helpful to be familiar with pinyin; that is, the system of phonetics using the Roman alphabet.

In Standard (Mandarin) Chinese, each Chinese character stands for one syllable: e.g. 上海 is ShangHai; 红桥火车站 is HongQiao HuoChe Zhan — Hongqiao Railway Station.

Photo by Touann Gatouillat Vergos on Unsplash

Now let’s begin!

Lesson 1 — Matching syllables to characters.

Most of the Metro stops are named after the roads they are on.

衡山路 — HengShan Lu — Hengshan Road.

Remembering the rule one character-one syllable, you now have 衡 (Heng) 山 (Shan) 路 (Lu).

You may have also picked up that 路 means road, a useful character to know when navigating the city.

Lesson 2 — Matching English translations to characters.

In other cases the name of a Metro stop is translated fully into English.

人民广场 — RenMin GuangChang — People’s Square

Looking at the characters, you have 人民 (RenMin) meaning ‘people’ and 广场 (GuangChang) meaning ‘square’ or ‘plaza’.

Lesson 3 — Listening to the Metro loudspeaker

On the train, the most important information you need to know is what the next stop is, and when you arrive. The phrase下一站 (XiaYiZhan) means ‘next stop’ and 到了(DaoLe) means ‘arriving’ or ‘arrived’.

Keeping everything we have learned so far in mind, can you figure out what the following phrases mean?

下一站：衡山路。

人民广场到了。

The next time you ride the Metro, read the signs in Chinese closely, and keep your ear tuned to the Metro announcer. It will take some time, but before you know it you will be able to read quite a few characters and understand some spoken Chinese. And hopefully for you, Chinese will become less and less of an “impossible” language to learn.

好运！(HaoYun — Good luck!)

This article originally appeared on an old personal blog 2012 when I lived in China. It has been edited for clarity.
Level up with intermediate content
Tags: Language Education, Education, Product Development, Language Learning, Travel

Our users have been asking for more advanced content, and we’ve been working hard to build it for you. Recently, we launched the first few packs of our brand-new intermediate course for Spanish, with more on the way soon.

We’ll talk more about our educational approach in future posts, but being that our goal is to provide you with everything you need to go from beginner to advanced in one place, building out great materials for every stage of that journey is really key.

Our aim with all our courses is to give learners the context to understand not only how to use certain grammatical concepts, but also when and why to use one over another.

The Forge team loves learning languages, and we appreciate in-depth, accessible, and easy-to-understand explanations of new concepts, so that’s what we’re hoping to give to our users.

Though, it’s challenging to learn from reading alone, so alongside our reading lessons, we have plenty of drills to reinforce the concepts and lock them into your longterm memory. Understanding on an intellectual level is one thing, but really internalizing those concepts and making them readily available for use is another.

As always, we really appreciate your feedback on any of our content or features. If you have thoughts on how we can improve or simply want to connect with us, feel free to reach out via the contact form on our site.

More intermediate course content is coming soon, and keep your eyes peeled in the coming weeks for more topic-based packs as well, including content focused on sports, technology, and more!

If you don’t have the Forge app yet, you can find links to download it at forge.co.

Connect with us on Twitter and Instagram.
Ineffective Eavesdropping
By: Pmj Erickson
Tags: Hungary, Hungarian Language, Language, Language Learning

I must admit that I didn’t do much speaking today, though I tried listening to other people. I could only catch a few words. I couldn’t really do any proper spying. However, instead of hiding under my headphones during a long walk, I opted to expose myself to the sounds around me. I listened to people at shops, people chatting in the tram, people having a cigarette in this day of much-desired October sunshine.

The listening reminds me of words I recognize, but can’t recall. It gives me an opportunity to hear the sounds of the language, and to head home to look things up. This also keeps me connected to my original goal — learn enough to stop being so anxious. A foreign language can feel like a grand secret, but later you understand enough to eavesdrop and discover everyone is just talking about what they do in your native tongue: the price of things, gossip about so-and-so, things we have to do in a day, how to get where we plan to go. In any case, I have been listening, reading every written word I see, and staring at advertisements long enough to make people think the sign was a fascinating bargain.

I bought some decaf coffee in a shop and the woman at the check-out showed me that it was decaf, just making sure. “Igen. De sajnos,” I said. Yes, but unfortunately. I needed the decaf one. This made her suppress a grin. She kept that half-grin through the whole transaction. I need more things like this, short phrases or words that will set me at ease. We didn’t say much to each other, just a greeting, an acknowledgement of what was mine, wishing her a good day. These things that I couldn’t have done in the Spring, that would have left me blushing or mute, I am saying them now as though I’m just a slightly shy or reserved local. Or that’s what I tell myself since I can’t hear my accent.

-pmj
IMPORTANCE OF VOICE ACCENT TRAINING IN YOUR CAREER
By: Shradha Hrd
Tags: Accents, Language Learning

You may have a good knowledge of the English language i.e. vocabulary, endless grammar rules etc., still many a times, communicating becomes a struggle. Taking your English accent to another level is an easy and enjoyable task. It helps you leave a lasting impression on your listeners and boosts up your confidence while giving interviews. Also, you end up developing good personality if you possess great voice accent skills.

Apart from pronunciation and grammar, voice accent training gives you many other benefits. VOICE training talks about your pitch, vocal organs and articulation whereas ACCENT training deals with tone and intonation. Intonation is all about rise and fall in pitch, also termed as ‘speech music’. The change in pitch defines the tone of your speech.

Assessment of voice and accent is based on two parts, reading aloud followed by speaking. Reading aloud gives the idea of how articulate an individual is and speaking clears whether a person knows how and what to speak or not. According to voice accent experts, following the below guidelines can aid you with your voice and accent skills:

Listen to the native speakers attentively. Watch good movies or follow newsreaders to enhance communication skills and vocabulary. Control your speech rate. Keep reading dictionary to know meanings, spellings and pronunciation. Record your audio speeches and listen so as to realize your common mistakes. Listen to good and motivational audios and videos on internet over and over again and try to imitate them. Read newspapers and magazines are also helpful for grasping vocabulary knowledge. Do some MOUTH EXERCISES to enhance clarity in voice and accent. Last but not the least, enroll for a voice accent training program.

Voice Accent Training: It helps you learn and apply the English speaking skills in an effective manner. To apply an impact on the listener, it is very crucial to speak clear and with proper word stress. For this, your mouth needs some warm-up exercises. Get trained from professional experts and improve your English speaking accent.

Now while getting trained, you may need to focus on eight major points in detail:

The phonetics Jaw and articulation exercises Vowel and Consonant sounds Pronunciation Syllable and word stress Formation of sentences Voice modulation Speech therapy

An accent that is acceptable by all international community is called neutral accent irrespective of the listener whether he is Canadian, Australian, British or American. Speaking with your friends and relatives can be done in casual way but, officially you need to follow the neutral accent so as to be accepted globally. Most of all, FLUENCY and CONFIDENCE matter a lot and this comes with good practice and the time you invest in doing so.
The Onus is on You to Understand My Accent
By: Ray Zhong
Tags: Google, English, Immigrants, Diversity, Language Learning

Photo by Jon Tyson on Unsplash

If you have lived in a region where the primary language is not your mother tongue, you must have had a moment like this:

You pluck up all the encourage and answer a question in class;

You try to present a new idea to your manager in a time crunch;

You try to order from a menu filled with unfamiliar words, with a long line waiting behind you;

The professor, classmates, manager and waiter, seeming confused, distracted or unattentive, ask you, with a varying level of politeness.

“I’m not sure if I understand.”

The immediate thoughts that come to your mind are:

“It’s because I stuttered again.” “I wish my English were better.” “It must have been my accent — I need to enunciate better.” “It’s my fault.”

But is it your fault?

In my first two years in the US, I lived and breathed that belief — if I wasn’t being understood, it’s my fault — my accent, my grammar, my wordiness, my limited knowledge of cultural references.

The longer I live here, the less I feel the onus is on me.

Born and raised in China, I started to learn English in 7th grade, like most Chinese people of my generation. I spent lots of time learning English and it paid off.

At the age of 19, I could pay my college tuition by tutoring TOEFL — a standardized test to measure the English proficiency of non-native speakers — at a language school.

I won Best Delegate in Model United Nations Conferences with attendees from 30+ countries and English as the spoken language.

I got promoted as one of the youngest managers at my first job, in a US company where English is the working language.

I got a near-full scholarship for a top MBA program in the US and I have a GMAT of 770.

The whole point of my shameless displaying trophies is that my English is good enough to get around, personally and professionally, both in the US and in China.

Do I write in impeccable grammar?

Of course not. You might have spotted 100 grammatical errors in this article by now; and that’s even after Google Doc did its diligent correction.

Do I speak with an accent?

YAS! 100%!

I’m as happy with my Chinese accent as Sofia Vergara with her Colombian accent or Heidi Klum with her German one.

My queen!

But it took years for me to grow the confidence that I can hammer my message home perfectly even with a less-than-perfect English and a non American accent.

Even though, along the way, I kept receiving comments, explicit or implicit, about my accent.

Let me share two of such comments. Profiles of the people mentioned are tweaked for the purpose of anonymity.
Migrating Elasticsearch Clusters Using Logstash
By: Ajay Raj Nelapudi
Tags: Logstash, Data Migration, Elasticsearch

A guide to migrate Elasticsearch clusters using Logstash

Introduction

In order to migrate Elasticsearch clusters from cluster A to cluster B, be it from on premises to Amazon Elasticsearch Service or from other hosted Elasticsearch Service, taking a manual snapshot on cluster A restoring it in cluster B is usually the recommended method and is widely used. However, in a few cases where manual snapshot and restore would not be possible, say due to version mismatch, then Logstash could be used.

In this article, I have listed all the steps I have followed to migrate an Elasticsearch cluster from self managed Elasticsearch cluster to Amazon Elasticsearch Service.

Steps

1. Installing Logstash: You would require an EC2 instance or a local machine with access to both the source and destination cluster. In my case, I have launched an EC2 instance of type m4.large and setup logstash on it. Here are the commands I have executed.



$ yum install java-1.8.0-openjdk.x86_64

$ cd /home/ec2-user

$ curl -L -O

$ sudo rpm -i logstash-oss-7.0.0.rpm $ sudo su$ yum install java-1.8.0-openjdk.x86_64$ cd /home/ec2-user$ curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-oss-7.0.0.rpm $ sudo rpm -i logstash-oss-7.0.0.rpm

2. Installing Plugins: While Logstash comes inbuilt with the logstash-output-elasticsearch plugin, you would require the logstash-input-plugin to read documents from your source cluster. Here is how I have installed the same plugin.

$ sudo /usr/share/logstash/bin/logstash-plugin install logstash-input-elasticsearch

3. Create a .conf file: You would have to create a configuration file describing the pipeline for your Logstash instance. Here are the simple steps I’ve followed.

$ cd /etc/logstash/conf.d

$ vim logstash.conf

Here the configuration I have used.

input {

elasticsearch {

hosts => "<source host>:<port>"

index => "*"

size => 1000

scroll => "1m"

ssl => true

codec => "json"

docinfo => true

}

}





output {

elasticsearch {

hosts => "<destination host>:<port>"

index => "%{[@metadata][_index]}"

ilm_enabled => false

ssl => true

}



stdout {

codec => rubydebug

}

}

4. Run your Logtsash instance as follows.

$ sudo /usr/share/logstash/bin/logstash -f logstash.conf

If everything is setup properly, you should find the migration happening from cluster A to cluster B and the relevant logs being printed to stdout.
Part 2 - Queries & Operations in ElasticSearch - Indexes & Documents
Tags: Elk, Kibana, Database, Search Queries, Elasticsearch

Quering the Data in ElasticSearch

For this part, I have created these 5 documents with id 1 to 5 in index “shirts” having different properties/fields.

You can see the information about index “shirts” including total documents etc and display some documents using _search API in elasticsearch.

For this you can query :

GET shirts/_search

Fetch all the shirts of brand ‘gucci’

The query will match the brand (field-name) of all the documents and fetches those documents whose brand matches with “gucci”

2. Fetch all “red” color shirts of “zara”

For this query we need two match statements. We can add match statements in this way:

As we can see our bool query returns all such documents whose brand must conatins “zara” and color must contain “red”.

we can also use “must_not” property instead of “must” for the complete opposite behaviour as that of “must”.

3. Fetch all shirts whose color is either “red” or “blue”

Here is the same query but instead of “must” we have used “should” property.

As you can see from hits.total.value we got 4 documents satisfying the above property.

4. Fetch all “non-red” shirts of “zara” or “gucci”

For this type of query, we can use multiple statements in this way

Dealing with numbers

5. Find all the shirts whose price is more than 250

For this we can use “range” query like this

6. Fetch all the “medium” sized shirts whose price is between 150–350
Choosing the Best Database in a System Design Interview
Tags: Mongodb, System Design Interview, Cassandra, Hive, Elasticsearch

Choosing the Best Database in a System Design Interview Awadhesh Jan 21·2 min read

For any scalable System Design, the choice of DataBase is very Important. Database selection will not impact Functional Requirement But it can hugely impact non Functional requirements like Scalability, Performance, and so on.

All the Database has been designed to keep in mind of Special Requirement. There is no Replacement for each Other, But Every Database having its own expertise Area.

Choice of database Depends on So Many Factor But below 4 factors is mostly Helped us to choose proper database :

1. Data is Structured or Non-Structured

2. Query Pattern

3. Amount of Data Need to Handled.

4 Consistency, Availability, and Partition

Data Schema:

if our data schema is fixed, means we know all column of a table then we can choose SQL AS well NO SQL database, But if every record having

A different set of Attributes then SQL DataBase will not be suitable.

ACID :

if we want to achieve ACID property like transitional data then we should choose SQL Database(like MYSQL)

Amount of Data :

if we have a huge amount of data and if we want data should be distributed in more than one region then the Cassandra database is the Best Choice. we can use Mongo DB also.

But Reading Data from Cassandra is fast compare to MongoDB due to their Architectural Difference. Mongo DB follows master Slave Architecture But Cassandra follows peer to Peer Architecture.

Caching :

If we want to cache the Data for better performance then we should use Redis or Memcached. Choice Between Redis and Memcached will depend on Many Factors. I will write Another Tutorial on that

File Storage:

we don’t store file data in the database. We need some Blob Storage. Amazon S3 is the best choice to Store files. We can use CDN (Content Delivery Network also) along with Amazon S3 for serving files fastly.

Text Search :

Sometimes we need to support text search or maybe fuzzy search. In this, we can use Elastic Search or solr as a Database to store data. Search time will be faster.

Time Series database :

This database is suitable when we are building a Monitoring Based System. We normally append our data like a log file.

for accessing data we Query based on range. so this database is built-in functionality to support range based Query.

Example of Time Series database like influx DB, Open TSDB. we can use any one of them.

Analytics Database :

for Analytics use Case we can use HIVE Database. this will help because it’s running on top of Hadoop. If Our data already sitting in the Hadoop file system, then it will be easy to Query through Hive Database.

I hope this will give some Overview to choose a proper database. I will write Another Story for detailed Explanations for each database.

Happy Learning !!!!!!!!!!!!!!!!!!!!!
Elasticsearch: achieving the analytics dream
By: Chetan Puttagunta
Tags: Elasticsearch

Elasticsearch: achieving the analytics dream [Originally published June 5, 2014]

Analytics infrastructure has never been easy. As enterprise investors, we know this all too well — as data is generated from various applications and stored across disparate systems and databases, the barriers to translating all that data into insights can be massive. The rapid growth of data volume, variety, and velocity has only sharpened this pain point, and it’s no secret that delivering on the promise of analytics has proven to be an extraordinary challenge.

Data scientists and analysts need to spend less time cleaning and extracting data and more time getting at the underlying insights. That’s why we’re so excited to lead Elasticsearch’s $70M Series C financing — because they have proven to be in the very early innings of showing the world what’s possible when real insight can be derived from all types of data.

Elasticsearch is the company behind Elasticsearch, Logstash and Kibana — a unique end-to-end analytics stack that has truly changed how enterprises can think about their data. Founded by open source veterans Shay Banon and Steven Schuurman, Elasticsearch and its ELK stack have resonated with developers and enterprises to become one of the fastest-growing open source projects with over 8M downloads since launch. Beyond the rapid uptake of Elasticsearch, the broad range of use cases is particularly impressive.

From early stage startups to some of the biggest companies in the world, enterprises are using Elasticsearch to understand their data like never before. One Fortune 500 Company is processing millions of events per hour across multiple business lines through Elasticsearch to enable strategic decision making at speeds that are simply unprecedented. Another is processing nearly 1.5M log events per second to make sense of how their customers are interacting with their products in real time.

We estimate the analytics infrastructure market dominated by the likes of Oracle, SAP, and IBM to be nearly $15B annually. However, all of this spend has not helped enterprises quickly and efficiently cross the chasm between data and insight. This gap represents one of the biggest opportunities in today’s software landscape. We believe, along with the thousands of enterprises already using this powerful analytics, search and logging platform, that Elasticsearch is in the position to revolutionize this market.
The basics
By: Franco Martin
Tags: Logging, Kibana, Monitoring, Elk Stack, Elasticsearch

In this series we will set up a basic three node Elasticsearch cluster to hold our data and use Elasticsearch’s built in redundancy and scalability features to make our cluster perform adequately and allow some failures to be tolerated.

Secondly we will install Metricbeat and Filebeat to generate some data and Kibana for visualizing our Elasticsearch data and to create dashboards and alerts.

In the third installment of this series, Ill guide you through the steps to create index templates and lifecycle policies so you can keep your cluster running smoothly.

Finally, we will set up alerting so we can be notified when something is wrong.

During this series we will be using Open Distro for Elasticsearch (ODFE). This distribution is based on the open source version of Elasticsearch and is fully open source too. It has several features that make it ready for its deployment in corporate environments like alerting and LDAP integrations for authentication and authorization.

To begin, let us get some definitions out of the way:

Node types

A node is an instance of Elasticsearch, it can be a physical machine, virtual machine or even a docker container. There are several types of nodes:

Master: Manages the overall operation of the cluster and keeps track of the cluster state.

Master-eligible: Nodes that can be elected as the cluster master.

Data: Performs all data related operations.

Ingest: Preprocesses data before storing it in the cluster.

Coordinating: Delegates client requests to the shards on the data nodes.

Indices

An index is a logical namespace that points to one or more shards that store your data. The equivalent of an index in the SQL world is a database.

Shards

A shard is a single Lucene instance. It is a low-level “worker” unit which is managed automatically by Elasticsearch. What you’ll need to know about sharding is that they help with performance by splitting data into pieces and storing it in different nodes and they can also be leveraged to achieve high availability by replicating shards in different nodes so that in the event of a node failure either the primary or a replica shard will be available.

Next Steps

In the next article we will get started with creating the elasticsearch cluster.

See other related articles.

Other useful information

ELK Terminology

Clustering

Open-Distro-for-Elasticsearch

Elasticsearch
Using Observability Tools with Envoy
By: Mike Dunn
Tags: Envoy Proxy, Kibana, Elasticsearch

Photo by Alex Eckermann on Unsplash

This is the second Envoy & Open Policy Agent (OPA) Getting Started Guide. Each guide is intended to explore a single Envoy or OPA feature and walk through a simple implementation. Each guide builds on the concepts explored in the previous guide with the goal of creating a very powerful authorization service by the end of the series. Be sure to check out the first guide of this series on Medium.

While our solution is still very simple, it is a great time to show how to make our solution observable with log aggregation. This makes it easier to think about how to scale and productionize our solution. As we start to develop and apply authorization rules at scale it will be handy to have all of the logs aggregated and displayed in one place for development and troubleshooting activities. In this article we will walk through how to set up the ElasticSearch FluentBit, Kibana (EFK) stack to pull your logs together from all of the docker containers in your local development environment.

All of the source code for this getting started example is located on GitHub. → Envoy & OPA GS # 2

SOLUTION OVERVIEW

The solution that we will build in this blog is shown below. We will send docker logs into an EFK stack that is also running inside docker. Each of the containers in our solution simply send logs to Standard out and / or Standard Error. No agents nor other special software is requirements are imposed on the observed applications.
MovieBuzz System Design: Coding End To End System From Scratch!
By: Bhushan Gosavi
Tags: System Design Project, Kafka, Kubernetes, Cassandra, Elasticsearch

Database Choice

We want to store 10 million user details and 5 million movie details. We are looking for a highly available database. We can compromise with the Consistency of user details and movie details. The best choice for storing this big data is Cassandra.

When the user opens the application we want to show a list of running movies in the user’s city. Once a user selects a movie, we want to show the user a list of the nearest Theatres in the user’s city running the shows for given movie. We can’t achieve this using Cassandra. We need a search engine for that. We can use ElasticSearch to solve these use-cases.

Cassandra is designed for heavy writes. As any write operation is just adding data to Memtable in RAM and appending data to commit log in the target node. So we can store all the movies and theatres details in Cassandra. The read operation in Cassandra is performance intensive. As reads have to go through n number of SSTables on disk via multiple caches in memory and disk. So we should try to avoid reading from Cassandra as much as we can.

Whereas for ElasticSearch, write operation is costly as every time we insert a document in ElasticSearch, we are indexing that document. So we should only store Movies and Theatres fields on which we want searchability. We are not allowing users to search movies by Actor names, so we should not store Actors associated with movies in ElasticSearch.

We can’t compromise on consistency in case of which seats are available and we don’t want multiple users to book the same seats in the same theatre. So we need a highly consistent relational database even if at the cost of availability. For this use-case, we can go with the sharded SQL database.
Cross-chain Asset Transfer Based on MAP Protocol
By: Map Protocol
Tags: Map, Blockchain, Map Protocol, Marcopoloprotocol, Crosschain

Since the birth of the Bitcoin in 2009, thousands of public chains have been produced by using blockchain technology. It is obviously that there is a growing desire to link these blockchain network together. This leads to the chain interoperability research. MAP protocol is an open, fully decentralized chain-to-chain interoperation protocol that enables the interoperability of multiple blockchains.

Currently MAP protocol has successfully launched a proof of concept that enables transfer tokens between two different blockchain network. This demonstration has been deployed in two blockchain network: Mouse and Duck. Through this solution, users could move their token from one blockchain to another easily. We will make some technical introduction here.

We assume their is a user has certain amount of Mouse token in mouse chain. He want to move some of his Mouse token to Duck chain. First he need to sign a cross-chain transfer transaction in Mouse, this transaction would lock some of his mouse token into a locking address. And once this transaction is confirmed. The miner in the Mouse chain would broadcast the crosschain request to Duck chain. Any miner in Duck Chain receives this request could use the ULVP module(detail of ULVP module could be found here) to verify the locking transaction in the Mouse chain. There is a reference contract in Duck chain which could deliver the verification result to the mint contract. If the locking transaction is indeed confirmed in Mouse chain, the mint contract would mint Mouse token to the designated address. If the locking transaction in not confirmed, then nothing would happened in Duck chain. The process of cross-chain token transfer procedure could been found in the picture below:

1. If you want to try this token transfer demo, you can use this link.

First you need to install Metamask (See instruction of Metamask Here ) in your browser. And add config of mouse and duck network into the Metamask.

Name for Mouse: MOS, Name for Duck: DUCK

URL for Mouse：https://rpc.mapscan.io/ , URL for Duck : https://rpc.mapscan.io/duck/

Chain ID for Mouse :1000, Chain ID for Duck :1001

2. Then you can add Mouse or Duck token to you metamask. The contract address for MOS and DUCK in each chain are the same: 0x40FB23023B8Bb73a19e949FA00ef3CCF7EE07Bb6 .

3. After that you could create two account in each chain.

4. You can request faucet to issue initial token in Mouse or Duck to you new created account.

5. Once you received the token, you can start transfer it from one chain to another.
Pelagios in the Indian Subcontinent:
By: Katherine Bellamy
Tags: Annotation, Maps, Linked Data

With such a large number of annotations, we decided to focus disambiguation on the Carnatic and Coromandel coasts on the south east of the subcontinent. 335 toponyms that could not be assigned automatically using Recogito’s in-built gazetteers were investigated manually, primarily using Google Maps and mapmyindia.com. Of these, 232 were successfully disambiguated and identified, extending the gazetteer for the subcontinent.

Map showing all automatically and manually assigned placenames from all annotated historic maps.

The full gazetteer created by the project is available here. Metadata is provided on the second sheet for the eight spreadsheet columns. The gazetteer includes a transcription of the toponyms as they appear on the map, universally unique identifier, latitude, longitude and tags assigned to describe annotated entity (e.g. settlement, italics, underlined). Time-management required a limit to be set for each disambiguation. A maximum of 15 minutes was spent assigning co-ordinates for toponyms. A designation of confidence for the disambiguated toponyms is provided, 1 for definite and 0 to indicate a degree of uncertainty. Colonial contexts present considerable complexity for disambiguation; some toponyms were untraceable. Around 1000 settlement toponyms could not be assigned locations either using Geonames or through disambiguation due to both the complexity of the corpus, and time constraints. As the database develops through different projects, it is hoped that coordinates can be assigned for many of these toponyms in the future.

A range of additional associated place-names were identified for 207 of the 232 disambiguated toponyms. These associated place names illustrate the complexity and potential of place name analysis. Some of the place names were similar, varying only due to transliteration. The associated names for Pedapalli, now the name of a district of northern Telangana, were clearly variations of the same root Petapouli, Peetapolee, Pettapoli, Pettepole and Peddapalli. Others showed far greater variation in nomenclature. The place located as Caliatur, for example, had six associated place-named: Krishnapatnam, Kistnapatnam, Caleture, Caletur, Calitore and Calletur. The Hill Fortress at Kondaveedu appeared on three maps and was associated with different names on each.

Kondaveedu Fort as it appears on three of the maps: ‘Condavera’; ‘Kondehuir’; ‘Condavir’

This preliminary work indicates the importance for smaller-scale explorations of place names that will map nomenclatures at the local level and trace their relationship to changes in government, administrative, urban growth and environmental transformations.
Our Bodies Are Maps
Tags: Poem, Writing, Travel Writing, Poetry, Maps

Our veins roads

to our homes.

Our eyes windows

to our fractured souls.

Our hearts doors

to the love we keep

trapped inside.

The love we feel

like we don’t deserve.

All the pain and love

and suffering and joy

lead us down paths

never taken,

roads abandoned,

trails uprooted.

I wonder where

life will take me.

If I’ll end up at a meadow during sunrise,

light escaping over the barrier

of the hill before it

freeing itself from night’s tight grasp

or a fire burning the night away,

all of the stars

falling from the sky

as the smoke envelopes us

into a realm of uncertainty.

Each of our bodies tells a story

left unspoken.

Fingers scribble words

onto our backs

that imprint into our skin.

Our feet leave trails

of stories behind wherever we go.

Our names hold our truths

only unlock-able by trust

and empathy

and love.

Our bodies are maps.

Our veins roads

to our homes.

Our eyes windows

to our fractured souls.

Our hearts doors

to the love we keep

trapped inside.

I wish to know your story.

Memorize every road

and path

and trail.

Scribble words that

imprint into your back.

Unlock your hidden truths

as we click together

like lock and key.

Be apart of

your story

as you will be apart

of mine.

A rock for me to lean on

when I’m tired.

A wall to hoist me up

when I can’t walk.

A ventilator to help me

when I can’t breathe.

A supporting character

always there,

always welcome, and

always real.