fabiochiu/medium-articles huggingface dataset:
Developer Spotlight: Sumit Kharche in the Cosmic JS Community
By: Carson Gibbons
Tags: React, Web Development, JavaScript, API, Developer

This blog was originally published on Cosmic JS.

In this installment of the Cosmic JS Developer Spotlight Series, we sat down with Sumit Kharche, a Full Stack Software Developer residing in Pune, India. Sumit is an active member of the Cosmic JS Community, having recently built the new React Static Blog, which is available in the Cosmic JS Apps Marketplace. With more community projects on the way, we’re excited to interview one of our own for this Spotlight. 😎 Follow Sumit on Twitter, LinkedIn and GitHub, and enjoy the conversation.

Cosmic JS: When did you first begin building software?

Sumit: I started building software when I was pursuing my Bachelor’s degree in Computer Science back in 2011.

What is your preferred development stack?

I have really enjoyed building projects using the Microsoft .Net technologies stack. I’m currently working on a team that uses React, Redux, and Material-UI on the front end and .Net core as backend. Recently though, I’ve found myself really enjoying working in a JAMStack. On the client side, I build static markup with React using React-Static, Gatsby and then provide APIs with Cosmic JS.

What past projects are you most proud of and why?

I built a JAMStack website by myself in couple of days. I chose React-Static, which is completely new for me, as was Cosmic JS. It was very challenging but I learned so many incredible skills as a result. Take a look at the demo.

You’ve submitted apps built in React Static, Svelte and more. How are you finding these new frameworks and technologies?

I love exploring new development stacks and expanding my knowledge. I always love to read about the new frameworks and technologies. Dev.to, Medium, Twitter, etc will always help me in finding the new technologies and also keep me updated. Now, because of the way Cosmic JS simplifies my work, I love to integrate it with different frameworks.

What are some technologies you are excited to learn more about?

That’s a big list. There is so much cool stuff out there I’ve been wanting to get into. Currently, I am excited to explore more about Cosmic JS. Also, I have been eager to get some time to spin up on .Net Core. I am excited about serverless stacks.

The Cosmic JS Spotlight Series is dedicated to showcasing developers that are building apps using modern tools. Learn how to contribute here. To stay connected with us follow us on Twitter and join the conversation on Slack.
There Is No “Safe Amount” of Cigarette Smoking
By: Dr. Hesham A. Hassaballa
Tags: Smoking, Vaping, Health, Medicine, Research

There Is No “Safe Amount” of Cigarette Smoking

New research indicates that even a few cigarettes a day can cause extensive damage

Photo by Sajjad Zabihi on Unsplash

There may be a perception among many in the general public that, while heavy smoking is recognized as a problem that causes lung disease, smoking “a little bit” should be OK. “I only smoke a few cigarettes a day, Doc,” is what I have been told by many patients. Well, recent research has shown that even this can cause extensive, lasting damage.

Researchers studied data on six US population-based cohorts included in the NHLBI Pooled Cohort Study, looking at smoking rates and the decline in lung function. After studying over 25,000 patients, they found that, even if people stopped smoking years ago, lung function continues to decline at a faster rate than never-smokers.

Now, everyone experiences a decline in lung function with age, but this decline is greatly accelerated in people who smoke. Even in people who smoke less than 5 cigarettes a day, their lung function declines at a rate of around five times that of former-smokers.

The researchers concluded:

Former smokers and low-intensity current smokers have accelerated lung function decline compared with never-smokers. These results suggest that all levels of smoking exposure are likely to be associated with lasting and progressive lung damage.

The study was published in the journal The Lancet — Respiratory Medicine on October 9. 2019.

In other words, there is no “safe” amount of smoking. Even smoking less than 5 cigarettes a day can cause lasting lung damage. And when the lungs are damaged, they are frequently permanently damaged.

Given all the bad things about vaping, one would think smoking is better. As a lung doctor, I would say, “Umm…no.”

“Well,” someone may say, “it is better than vaping,” given all the horrible things that have been coming out about vaping. Indeed, I have been quite vocal against vaping. As a lung doctor, however, I would have to say, “Umm…no.”

The lungs are beautiful, exquisite organs. They are very sophisticated, and they are excellent at taking the life-sustaining oxygen from the air and transferring it to the blood, where it goes to the rest of the body.

That said, the lungs were only designed to process air — not cigarette smoke, not air pollution, and certainly not vaping chemicals/oils. Inhale anything into the lungs other than air, and you are placing your health at risk.

The opinions expressed in this post are my own and do not reflect those of my employer or the organizations with which I am affiliated.
Infographic: 2016 Nobel Prize Winners
Tags: Data Journalism, Infographics, Nobel Prize, Science, Data Visualization

Infographic is a visual representation of data intended to present information clearer and more engaging way.

Follow
The Music Plays On — Mahler Symphony №4
By: Donato Cabrera
Tags: Mahler, Donato Cabrera, Las Vegas Philharmonic, California Symphony, Music

Gustav Mahler, c. 1900

I’ve decided to honor the decision by the Royal Concertgebouw Orchestra and the Royal Concertgebouw to have an online version of the Mahler Festival by offering my favorite performances of the symphony that the festival is broadcasting each day.

Gustav Mahler ascended to the throne of the conducting profession in 1897, when he became the General Music Director of the Vienna Court Opera, now the Vienna State Opera. It was music’s most prestigious position and with it came an enormous responsibility. Mahler was everywhere and ran the opera house with an iron fist. Modern stage direction, lighting, set design were all in their infancy in opera and Mahler held a tight grip on all of it!

His fourth symphony, composed during the summers of 1899 and 1900, was the first he composed after being hired in this position, and it’s interesting that the subject matter and general mood of the symphony is in direct opposition to the hectic conducting life he had at the time. Also, rather than working in a little ‘hut’ behind a small hotel at Steinbach am Attersee, Mahler could now afford to rent a villa. He chose the Villa Kerry, a 30-minute walk above the the lake, Altaussee. Unfortunately, the weather for most of that summer was cold and wet and it turned that this villa was in earshot of the local bandstand. He was only able to compose part of the symphony at the very end of his holiday symphony.

Villa Kerry, Altaussee

The following summer (1900) was spent in his newly built composing hut on the property where his villa was being built, in Maiernigg, a tiny village on the shores of the Wörthersee.

Mahler’s ‘composing hut’ in Maiernigg

Like Beethoven’s Symphony №4 in relation to his Symphony №3 ‘Eroica,’ Mahler’s Symphony №4 is restrained and classical in size and structure compared to his gargantuan Symphony №3. It has the standard four movements, but it retains one special characteristic that was unique to the previous three symphonies — there is a vocal soloist. For the last movement, Mahler uses Das himmlische Leben (The Heavenly Life), a song that he wrote in 1892 based on a poem from the collection, Das Knaben Wunderhorn (The Youth’s Magic Horn). For a wonderful English translation, please of the poem, please go here.

The first three movements, while not having a text, certainly help form a narrative of a child’s view of the world. Indeed, all four movements contain themes that resemble children’s songs more than anything else. Here are my favorite performances.
How to use Silk to publish data-driven stories efficiently
Tags: Infotimes News, Journalism, Data Visualization

Silk and InfoTimes have partnered up to offer a series of free webinars and hands-on sessions to journalists who want to hack their data skills.

To sign up: register here. http://goo.gl/forms/LJnwvSqjyI

The webinars will teach journalists how to master the data publishing platform Silk. With Silk, data journalism is easy. And you can go from spreadsheet to analysis to publish-ready visualizations in minutes. Without the need to code a single line. What’s more, Silk is free to use.

Alice Corona’s webinar

The webinars will be held by Amr Eleraqi, founder and director of InfoTimes, and Alice Corona, Silk’s in-house data journalist.

The first webinar will show “how to use Silk to publish data-driven stories efficiently”. At the end of the webinar, we’ll ask users to come up with a dataset they’d like to see visualized.

The following webinar will look at how to transform a selection of these specific dataset(s) into a Silk project, with ready-to-publish visualizations. Users can either follow the chosen case study or apply the steps to their own data project. Silk’s data journalist will be available also after the webinar, to offer personal assistance to those who wish to continue with their projects after the duration of the training.

To sign up: register here. http://goo.gl/forms/LJnwvSqjyI

Schedule:

Thursday January 21st 2016, Introduction to Silk

Thursday January 28th 2016 Publish your own story with Silk

Please also invite friends and colleagues because they might learn useful skills from this webinar.

Hope to see you there!

Alice from Silk.co and Amr from InfoTimes

About Silk: Silk is a place to publish your data. Each Silk contains data on a specific topic. Anyone can explore a Silk and create interactive visualizations. Silk has been successfully used by The Guardian, The Washington Post, Mashable, IJNEt and Human Rights Watch. Here’s a 80 second introduction on Silk https://www.youtube.com/watch?v=JOJ5cXzEzf4

About InfoTimes:

InfoTimes are a modern information design and data visualization agency in heart of Egypt with one common philosophy: data should be designed for sharing.

Our mission at the current time is to improve the skills of Arab journalists, raise the awareness among Arab press with the importance of data journalism, how they can use the data sets and spreadsheets for their own good.
Are you REALLY automating your infrastructure?
By: Dan Siwiec
Tags: AWS, Terraform, DevOps, Infrastructure As Code, Continuous Integration

The project at hand was subjected to both cases above — the AWS platform was a big shift for the application and required rethinking its architecture. Rapid prototyping was necessary to make quick progress but experimenting with various architectures left a big mess behind. Also, the team was relatively new to infrastructure automation and unintentionally leaned towards manual changes which sometimes slipped our minds before they made it into the scripts.

To address these problems, the team adopted an approach that, at first, felt overly radical — we would periodically completely wipe our AWS account clean. It is important to make a distinction between this and performing a terraform destroy which only removes the components explicitly managed by terraform. This would miss the exact point of this operation — identifying the areas which were not yet automatically provisioned. Our nuke of choice was an open-source, well-maintained tool named aws-nuke.

The first time we did this, our hands were shaking as we were confirming this very destructive operation, which the tool requires you to do twice, for safety measures. The result of this was disastrous — an attempt to reprovision the application with our automation scripts failed with multiple incomprehensible errors. Recovery took a better part of the week and team morale was low — it felt like we just shot ourselves in the foot and took a huge step back. However, what was not immediately clear, was that in the process of rebuilding what we just destroyed, we would benefit in two ways.

First, we discovered all the tweaks we’ve made and haven’t automated — some of them we completely forgot about, others were simply not documented. The nuke operation identified them so that we could make an explicit decision about automating these changes or documenting them.

The second lesson came after we performed the nuke again — this time with a bit more confidence in our automation. Applying the manual changes, now well documented, for the second time and later third and fourth, we experienced the pain of manual configuration first hand and following the principle of frequency reducing difficulty and the old CI/CD mantra — if it hurts, do it more often — we were strongly incentivized to double our effort to automate them.

If it hurts, do it more often and bring the pain forward.

— Jez Humble, Continuous Delivery

Over time, we performed the nuke multiple times and eventually, it ceased to stress the team. We grew an appreciation for it as another tool in our toolchain, that helped sharpen our saw. It increased the quality of our automation, as well as solidified our confidence in the process, in very much the same way frequent production deployments do in the traditional CI/CD approach. We eventually incorporated the nuking into our regular stakeholder showcases and finally, those became pretty uneventful and boring. The good kind of boring 😉

Happy nuking, everyone!
How to Receive Water Temperature Data in Real-Time Using Raspberry Pi and Python
By: Deborah Kewon
Tags: Python, Data Analysis, Raspberry Pi, Data Science, Programming

VII. Access your Raspberry Pi using Terminal

Now that we have our Raspberry Pi IP address, we can access it.

Go to your terminal and paste the following script.

ssh pi@xxx.xxx.xx.xxx *your raspberry pi IP address*

Terminal will ask you for the password. The default password for Raspberry Pi is “raspberry”.

When you see “pi@raspberrypi” in green, use the following script to configure.

sudo raspi-config

Let’s enable some items. When the below screen pops up, go to “Interface Options”.

Click “P2 SSH” and enable SSH.

It is always nice to see the desktop of our Raspberry Pi — Let’s also enable VNC. With VNC viewer, we will be able to remotely access/control our Raspberry Pi.
4 interactive Sankey diagrams made in Python
Tags: Data Visualization, Python, Physics, Website Traffic, D3js

Plotly has a new member of the Plotly.js chart family: The Sankey diagram. Allow us to introduce you:

A Sankey diagram showing changing voter views. The Python code to make this chart is in this Jupyter notebook: https://plot.ly/~alishobeiri/1591/plotly-sankey-diagrams/

Sankey diagrams were invented to chart energy flows (such as through a steam engine). As an example, you might start with 100% of a fuel’s energy on the left side of the diagram, then X% is lost to friction, Y% is lost to compressing the fuel, and Z% is used for propulsion — these losses are shown on the right side of the diagram. Connecting the left and the right sides are ribbons whose widths are proportional to the flow magnitude. This simplified example would show 3 ribbons for 3 energy flows, but practical Sankey diagrams can have dozens (or hundreds) of ribbons. Sankey diagrams are still used in science and engineering for charting energy flows.

Today, Sankey diagrams are commonly seen charting the flow of website traffic. Consider the case where 1,000 website visitors start on 3 landing pages, then trickle off to hundreds of sub-pages before they leave the site entirely. Analysts use these diagrams to understand which pages get the most traffic, how the traffic arrives, and where the visitors go afterwards.

Plotly Sankey diagrams come supercharged with a few extra features:

Hover labels — Add labels to sources, sinks, and flows so that your audience knows what they’re seeing when inspecting a complicated Sankey diagram.

— Add labels to sources, sinks, and flows so that your audience knows what they’re seeing when inspecting a complicated Sankey diagram. Like all Plotly charts, there are open-source interfaces to make Sankey diagrams in R , Python , or JavaScript .

, , or . You can track the flow of individual items through a Plotly Sankey diagram. For example, the screenshot below shows one of 400 drones highlighted through all stages of its lifecycle. This tracking feature could be useful in manufacturing: Imagine tracking thousands of battery cells through the stages of an electric vehicle manufacturing line. The Sankey diagram could start with the delivery of the cells at the factory and end when they are installed in a completed car.

Individual items can be tracked through the flow of a Sankey diagram. In the example above, We see a Predator MQ-1B drone became part of the Predator program, was sold to the Air Force, then ended up in Afghanistan. There’s even hover text describing the fate of this drone. See this full-sized chart on plot.ly: https://plot.ly/~alishobeiri/1367.embed

Without further ado, here are 4 interactive Sankey diagrams made in Python. The Python code to make these is in this Jupyter notebook hosted on plot.ly.
So easy.
By: Martino Pietropoli
Tags: Comics, Cartoon, Illustration, Music, Drawing

Did you find this article interesting?

If you liked it and if you feel like it, leave some claps to help whoever wrote it to pay his Lamborghini Urus.

Thank you!
Summer in the Bronx
By: Tre L. Loadholt
Tags: Creative Non Fiction, Bisexual, Childhood, LGBTQ, Music

Summer in the Bronx

And love, unspeakable love.

Photo by Kaique Rocha via Pexels

For what seemed like countless summers of my upbringing, I’d be shipped off to my family in New York, specifically, the Bronx, in order to find some semblance of relief from boredom and the murderous heat of Savannah, Georgia. I’d count the days toward the middle of May and flaunt my happiness to my friends as much as I could, however, I knew I’d miss them.

I knew I’d want to know what their days would entail without me.

In the summer of ’98, I had two crushes: Joel & Mackenzie. Joel was Puerto Rican & Black and Mackenzie was Jamaican but was raised in Queens for the bulk of her life. (Every other weekend, she’d visit her aunts and cousins in the Bronx.) I lusted over them— would do anything for the heat of their presence to sway my way, however, I was not out then, so Mackenzie could never know my true feelings.

I paraded around my Grandma’s neighborhood, tossing back coconut icies, running through fire hydrants, and staying out late in the park. Bronx heat was a bomber; a killer, if you will. We’d have blackouts that’d last for days and I would find myself yearning for the sunlight just to get a glimpse of Joel and his smile or Mackenzie and her long legs.

I used to think she walked on clouds and I wanted to know just how soft her steps were.

She’d call me “Tree” with a hint of her Jamaican accent slipping through and she’d ask me to turn the ropes when we played Double Dutch. And what a damn honor that was — what it did to and for my ego . . . *Mac wants me to turn again. Maybe she knows.* But I was just hella good at turning the ropes and going with her flow and although I wanted to flow with her in other ways, I settled for our daily games.

Joel came and went. He was fluid, like water. I couldn’t catch him and even if I could, my hands weren’t big enough to hold him. He’d slide through every single time. Enigmatic — that’s how I described him.

He would sit near me on the park swings and just talk. Just talk . . . He had a gold tooth and a fat herringbone chain and my Grandma used to yell from our fifth-floor window for me to “get my fast ass upstairs" and I always ran away from him. Authority was our downfall — I never truly felt his heat until I couldn’t have it.

Summer became my favorite season that year. It was the year I’d compare all others to. It was the year I searched for the heat I loved and the heat I lost. I often wonder how both of them are doing; if Mac still walks on clouds and if Joel is still hard to catch.

I wonder, sincerely wonder if they knew about my heat.
How to Stop an Attacker From Hurting you
By: Christina Vaughn, Nurse, Freelance Writer
Tags: About, Better Living, Health, Self Improvement, Life

How to Stop an Attacker From Hurting You

Use the human body to buy you time to run

Photo by Timothy Eberly on Unsplash

The city of Dallas, like many other major cities worldwide, has had a high incidence of deadly home invasions over the last several years and this continues to be a problem for the Dallas residents and police department now in 2020.

Many of these invasions occur with the residents of those homes at HOME at the times of the crimes.

Unfortunately, many of those residents are shot and killed in their own homes and it takes much effort with too little staff for the Dallas PD to stop the streak of terror.

We are all aware as American citizens of the tragedy of frequent occurrence of mass shootings in our country and violent crime that surrounds us and our communities every day.

As unfortunate as these facts are, the larger truth is that the hopeless state our communities are in where so many families and individuals are daily lacking food, shelter, and adequate healthcare has evoked criminal desperation in those most vulnerable with the notion to do so.

Rampant drug and sex trafficking and crime, in general, are also NOT at an all-time low as President Trump will proverbially tweet-claim.

I do not believe in living in fear, but that said, I do believe in staying privy to the most recent risks to my home, family and our lives…and I also believe in readiness.

In my years as a nurse and as a parent, due to the risks, I encountered performing home visits in many dangerous areas and having been the single parent to small children and no man in the house at the time, I have mulled over the prospect of having to hand-to-hand combat an intruder or an attacker.

I have also imagined having to circumvent tragedy to myself or one of my kids if we are involved in a mass shooting outdoors or in a public place.

I have created scenarios in my head that I feel keep help to keep me in check with the reality of this life and its many dangers.

There are many ways an individual can be attacked or threatened. Some of those ways are in public, on the street or in an alley, or at a public place where others are gathered, getting in or out of one’s car anywhere, arriving to a new acquaintance’s home or being in some other unfamiliar place, or even in your own home or walking to or standing at a bus stop.
ICO Alert Crypto Minute: June 27, 2018
By: Evan Schindler
Tags: Cryptocurrency, Crypto Minute, Blockchain, Facebook, Bitcoin

Facebook Loosens its Ban on Cryptocurrency Ads

As reported by CoinTelegraph, Facebook is allowing some cryptocurrency ads to be promoted as long as they go through an application to be approved. The ads were originally banned in January 2018. Facebook stated that the ads were “associated with misleading or deceptive practices.” The social media giant intends to verify their advertisers by requesting licenses and other information. For now, ICOs are still prohibited under the new policy.

Bitcoin Education Center Opens its Doors in Atlanta

Whether you’re a blockchain novice or an avid cryptocurrency enthusiast, the newly developed Atlanta Bitcoin Embassy, may be the right place for you. According to American Inno, the location was founded by Jeffrey Tucker and hosts courses aimed towards teaching the community about digital currency, cryptoassets, and related blockchain technologies. It also acts as an organizational hub, allowing Bitcoin veterans to gather and discuss the entire cryptocurrency ecosystem.

Leading VC Firm Andreessen Horowitz launches $300 Million Crypto Fund, Plans to HODL

The announcement by the top Silicon Valley VC firm confirmed that the $300M fund would be dedicated to Bitcoin and other cryptocurrencies. As reported by CNBC, the firm plans to “invest aggressively,” and “consistently over time, regardless of market conditions.” Chris Dixon, a partner at Andreessen Horowitz was quoted as saying “We’ve been investing in crypto assets for 5+ years. We’ve never sold any of those investments, and don’t plan to any time soon.”

The firm seems to have a very positive outlook on the future, and longevity of the cryptocurrency ecosystem. Despite Bitcoin’s lows this year and the market’s recent downturn, reputable top investors are continuing to remain bullish on crypto, disregarding what’s happening in the markets.
No More Mess in my Head Around Phrases Related to Identity in Computing
By: Daniel Rusnok
Tags: Azure, Software Engineering, Software Development, Identity Management, Azure Active Directory

Azure

No More Mess in my Head Around Phrases Related to Identity in Computing

What is Identity? Azure Active Directory is just Active Directory in Azure? Microsoft Graph is a Data Visualization Framework or What?

Credits of logos for Microsoft.

Identity

What is Identity in Cloud

Identity is a unique identification of an object. Such an object can be a human being, machine, or a combination of it. When we talk in the cloud computing context, identity means a set of properties about this object stored in the cloud's datacentre.

Identity & Access Management

Identity Management (IdM) and Identity and Access Management (IAM) is an interchangeable term in identity access management. So if you are reading about one, you are probably reading about the second term too.

IAM is a framework of policies that tell what users can do in their restricted area and what he needs from the user to operate properly. Such systems identify, authenticate, and authorize individuals or hardware applications to use restricted resources.

IAM exists in the world without the internet too. It appears in different forms. For example, the “Staff Only” label at doors in markets, id card pinned at employees suit, or doorman as a profession by itself. Even your dog protecting yard is some Access Management of your property.

Dog protecting an off-limits area.

Every IAM operates in its defined context. The context of IAM specifies the number of properties it needs from the Identity. For example, every patient has a folder with properties about his identity at the doctor's office.
MIT’s LeakyPhones Help You Interact with Strangers by Sharing Music at a Glance
By: Cameron Coward
Tags: MIT, Research, Social, Technology, Music

The Tangible Media Group within MIT’s Media Lab is focused on researching human interaction with the gadgets and gizmos around us, and how that technology affects social dynamics. Their research has led to a number of interesting developments that we’ve featured in the past here on Hackster, but this is one you’re either going to love or hate. Amos Golan’s LeakyPhones are headphones that let you share your music with a stranger at a glance — and that let them share their music with you.

Golan’s motivation for this project is rooted in our current social dynamics, or lack thereof. In the past, starting a conversation with a stranger was a simple matter of saying “hello” when you made eye contact for a moment. These days, many of us spend our time in public with our eyes glued to our smartphones and our ears covered with headphones. For many of you, that’s a good thing — you don’t want strangers talking to you anyway. But, there’s no denying the effect it’s having on spontaneous interactions.

LeakyPhones is intended to open the doors back up to that kind of interaction, but respectfully. The idea is that if two people wearing LeakyPhones come across each other and hold eye contact for a moment, they’ll begin to hear what the other person is listening to. The longer they hold eye contact, the more the stranger’s music starts to overcome their own. Hopefully, that will spark a conversation.

There are, of course, a number of privacy concerns here. Many of you are already cringing at the idea of a stranger hearing what you’re listening to, or you hearing what they’re listening to. That’s why LeakyPhones has four different operating modes. You can either choose to allow transmission of your own music or not, and choose to receive music from strangers or not. It allows for controlled social interactions with strangers, but only when you want it.
Web Scraping a Javascript Heavy Website in Python and Using Pandas for Analysis
By: Megan Dibble
Tags: JavaScript, Python, Pandas, Jquery, Data Science

I set out to try using the Python library BeautifulSoup to get data on the retailers that would be attending a market, as shown on this webpage: https://www.americasmart.com/browse/#/exhibitor?market=23.

What I found, however, was that the BeautifulSoup function (BeautifulSoup(url, ‘html.parser’)) only returned the header and footer of the page. It turns out that this website relies on Javascript to populate most of the data on the page, so the data I was looking for was not in the html tags.

To find the AJAX request that returned the data I needed, I looked under the XHR and JS tabs in the Network section of the Google Chrome browser (see image below). The credit for this idea came from this blog post: https://blog.hartleybrody.com/web-scraping-cheat-sheet/#useful-libraries. You need to hover over the “Name” fields and right click to copy the link address, which you paste in the code below. There are a bunch of options here, but I just tried them one by one to see if they retrieved the data I was looking for.

The Network tab on the Google Chrome Inspect menu

import requests

r = requests.get(url)

info = r.json() url = ‘ https://wem.americasmart.com/api/v1.2/Search/LinesAndPhotosByMarket?status=ACTIVE_AND_UPCOMING&marketID=23' r = requests.get(url)info = r.json()

For this website, the data was returned in a list of dictionaries. I had to play around with the indexes to extract the data I needed. Ultimately, I wanted to create a Pandas DataFrame with location information about every retailer, and then merge it with a narrowed down list of retailers, housed in a .csv file. The link to the full code is on my github: https://github.com/mdibble2/Projects/blob/master/Web%2BScraping%2Bwith%2BAJAX%2Brequest%20(1).ipynb

I used the df.merge() function to select only the rows of data from the website that matched the retailers listed in the csv file (i.e. an inner join). The drawback to this approach is that every retailer needed to be spelled correctly with the correct capitalization in order to match the primary key of my first table, the one generated from the website.

To check that there were not any misspelled retailers, I created a series from the retailer names in the merged dataset and another series from the retailer names in the csv dataset. I then merged these together and used df.drop_duplicates to see if any unique values remained besides those in the merged dataset. I found that I had misspelled or incorrectly abbreviated 9 retailers. Once I corrected these records, the merged dataset gave all the information I required.

Some take-aways from this mini project were:
Role of Big Data in Academia
By: Ella William
Tags: Data Science, Big Data, Analytics, Information Technology, Data Visualization

CAN ACADEMIA, RESEARCHERS, DECISION MAKERS AND POLICY MAKERS MANAGE THE CHALLENGES OF BROADER COLLABORATION AND PRIVACY TO HARNESS VALUE FROM BIG DATA?

For academia, researchers and decision-makers, and for many other sectors like retail, healthcare, insurance, finance, capital markets, real estate, pharmaceutical, oil & gas, big data looks like a new Eldorado:

Read More
Data storytelling - Black Lives Matter!
By: Oluwatosin Sanni
Tags: BlackLivesMatter, Data Science, Storytelling, Data Analytics, Data Visualization

Data storytelling - Black Lives Matter!

Introduction

#BlackLivesMatter was founded in 2013 in response to the acquittal of Trayron Martin’s murderer. Black lives matter foundation, Inc. is a global organization in the US, UK and Canada whose mission is to eradicate white and build local power to intervene in violence inflicted on Black communities by the state and vigilantes.

So many death have been recorded as a result of Police violence in US which has claimed many lives over the years and left the family of the deceased in lifetime agony. The core question is

Can the death rate be minimized or completely avoided?

Yes, it can. Data scientist can give insight and recommendation to solve this problem. This key question motivated me to give a shot at the publicly available data on police violence in US on Kaggle.

I have done some exploratory data analysis on the US police violence & fatal shootings against citizen between January, 2015 and August, 2020. This analysis will not only discuss about black race people but in fact all races living in the US. The intention of this analysis is to answer the outlined questions, to/show;

1. Which day, month and year death is highly recorded?

2. What is the percentage of age bucket (group) and gender affected?

3. What race, state, city and demography are most affected?

4. What is the proportion of flee or not flee and threat to the police or not?

5. What are the most used arms?

6. Black lives Matter, further insight on black race

1. Which day, month and year death is highly recorded?

Death analysis by day

proportion by day

The above visual clearly shows that mostly killings occur in mid week that is, Tuesday and Wednesday and decline towards the weekend and rise again at the beginning of the new week that is, Saturday and Monday.

Death analysis by month
How Will You Change This Holiday Season?
By: Randy Wolken
Tags: Leadership, Business, Self Improvement, Self-awareness, Life Lessons

Lesson to Learn

We get to control how we act and respond to the situation around us. It is our life, and despite how others act or events unfold, we do get to choose what we do.

It is not easy to be upbeat, joyful, and helpful to others when surrounded by pain, uncertainty, or sorrow.

We can choose to be a ray of sunshine during the storm. Doing so changes the situation. Doing so changes our outlook.

We can choose to be the joy that is so often lacking in our world during this holiday season.
Working on an Open Source Project as a Team
By: Becca Viner, Caitlin Chan, Mai Nguyen, Tania Lind
Tags: JavaScript, Open Source, React, Programming, Women In Tech

Working on an Open Source Project as a Team

Reflecting on our experience contributing to open source project Reactime

Photo by Javier Allegue Barros on Unsplash.

Now that our team has launched Reactime version 7.0, we have some time to reflect on our experience contributing to an open source project. Our team was incredibly productive despite working across multiple time zones and exclusively communicating virtually. The most important aspects of our successful team dynamic were pretty simple: empathy and communication.

More concretely, here is how we put these into practice.

Before diving into a project, we discussed everyone’s individual goals and searched for an application that we were all passionate about and would enjoy working on. We all love using React to build applications and wanted to deepen our understanding of the JavaScript library. Reactime was the perfect open source project for us to do that.

Our approach to it was pretty simple. At every stage, our main focus was the experience of the user, the user of our app, and the developer who would want to contribute to our extension. We wanted to improve the performance of the extension itself to make it easier for developers to spot bugs in their applications and to easily identify performance bottlenecks.
11 Phrases to Be More Persuasive Without Being Overly Pushy
By: Michael Thompson
Tags: Relationships, Business, Leadership, Self Improvement, Productivity

Recently I came across a statistic that caught me off gaurd. According to research performed by persuasion expert Daniel Pink in his book “To Sell Is Human,” 41 percent of the average person’s workday consists of influencing, persuading, and convincing the people around us.

This statistic may seem high, but it makes perfect sense. From being offered opportunities and getting your ideas to fly, to convincing your kids in an effective manner to pick up after themselves, persuasion is all around us and it plays a key role in both our personal and professional lives.

As an introvert with a stutter who has spent the last two decades as either an entrepreneur or in sales-based roles, I’ve studied the topic of influence and persuasion a great deal. I love the work Daniel Pink puts out. The same goes for the author of “Influence” and “Pre-suasion,” Robert Cialdini.

However, if I was looking for an introduction to persuasion, few books are more effective than “Exactly What to Say,” by international sales trainer Phil M. Jones. This is for the simple fact that Phil’s teachings are not only super easy to digest, but they are also extremely practical as he gives very clear examples as to why they are effective along with loads of everyday examples.

I’ve been sprinkling the phrases Phil recommends into my daily conversations since coming across his book three years ago. They work. Best of all, many of them you can begin to implement today without feeling like you’re being overly pushy.

This past week, Phil was kind enough to give me the green light to share some of his magic phrases. Below are 11 of them to get you started.
Kick Start Your Marketing Career: Take the Google Advertising Fundamentals Exam
By: Jordan Silton
Tags: Career Advice, Google, Certification

Do you know what all the ads are doing on the top and right side of the search results?

At this point, almost everyone uses the internet, and most internet users use search engines to navigate the web, find information, and purchase goods. The goal of most search engines is to deliver the most relevant results for each query. Do you feel that your company or product is relevant for certain search queries?

If you are trying to kickstart your marketing career, it is very important that you understand search engine marketing. In many organizations you will be hard pressed to find someone who is knowledgable about search engine marketing. Not only that, very few people can say they are experts. If you’d like to show your initiative and become an expert in one portion of online marketing, take the Google Advertising Fundamentals Exam.

According to the exam website, the fundamentals exam is designed to cover “basic aspects of AdWords and online advertising, including account management and the value of search advertising”. It’s not just a test on how to use AdWords. In fact, the exam only scratches the surface of AdWords and its functionality. The fundamentals exam is really a course on how online advertising works on Google. How do the ads get there? Why should I click on a paid ad? How can I know the ads are safe? Are the ads relevant to my search?

This AdWords exam is only one of the exams Google offers. They also have training and testing centers for Google Analytics and Google Apps. You don’t even have to go with Google. If you would rather get a certification from Microsoft, you could take the adExcellence exam.

Gaining expertise in search marketing can help differentiate yourself from other job applicants and employees in your future workplace. If you don’t want to pay the $50 for the exam, all the learning materials are available for free online. Take a look for yourself; I bet you’ll learn something new in the first 5 minutes.
Do White Men Rule the World Because They’re More Intelligent?
By: Paul Abela
Tags: Patriarchy, Society, Culture, Gender Equality, Racism

In 1921, Albert Einstein was awarded the Nobel Prize in Physics “for his services to Theoretical Physics.” Einstein was undoubtedly a genius and his contributions to science were deserving of such a prestigious award. Being awarded a Nobel Prize is no easy feat. It involves making an outstanding contribution to the sciences, economics, literature or world peace.

Now, genius is highly subjective. What genius means for some, is different for others. Not every genius who has made a brilliant scientific discovery receives a Nobel Prize — Nikola Tesla being a notable exception. And not every genius falls under the scope of the Nobel Prizes. There are countless brilliant creatives in the arts, music and film industry’s who aren’t awarded prizes, as the arts (bar literature) don’t fall under their remit.

But, the Nobel Prizes do offer a useful barometer into whose considered a scientific, economic, or literary genius deserving of praise, and whose not. When looking at the diversity of those awarded a prize the shocking lack of diversity may surprise you.

Since the first ceremony in 1901, 923 people have received a Nobel Prize. Only 54 prizes have been awarded to women or 6% of the total.

The lack of racial diversity might be even worse.

Of the 923 winners, 70 are of Asian descent. 14 winners are of African descent. And a black person has never won a Nobel Prize in Science, not one.

That means 90% of winners have been white, 94% of winners have been male. If you saw these statistics without reflecting on them, the only conclusion you could make is white men are geniuses, everyone else isn’t.
The popularity of Django
By: Projjal Gop
Tags: Programming Languages, Popularity, Python, Trends, Django

Django is a open-source framework of Python which is unlike any standalone server solution like Spring, Rest API and ASP.Net to name a few.

Like any other MVC framework, Django too uses model-template-view architectural pattern.

As the charter states:

Built by experienced developers, it takes care of much of the hassle of Web development, so you can focus on writing your app without needing to reinvent the wheel. It’s free and open source.

Model: A model in Django specifies the interface and the architecture for defining the operation of a particular operation based on a particular situation.

Views: Views contain methods for defining the operations done by a particular call to url requests.

Templates: Templates deal with rendering of the pages and define operations to deal with url response from particular view in a model.

The MVT architecture of Django is as depicted:-

Few useful commands:-

• django-admin startproject : Creates the project directory and inbuilt files of that project.

• python manage.py startapp : Creates an app under current project.

• Python manage.py collectstatic: Makes meta information of static files and migrates them under one static directory.

The project directory of Django is as shown:

manage.py contains the code adapter to handle databases, connections and templates of the current server-side solution.

Sqlite is the sql database which is the database for the given project.

The Django framework is not globally used in production servers as of yet but its success is enviable.

Do you know what powers YouTube and the Netflix platform so that its performance is such hyped?

Yes, it’s Django.

The current trends and the relative position of Django is as shown:

Trends in the popularity of programming languages

So why you don’t see such wide use of Django in production servers?

Well there are two primary reasons:

• Django is relatively new having launched in 2005 and not fully deployed until 2007.

• Programmers are reluctant to change to Django because of its relatively limited reach.

So what are the features of Django that is attracting developers like magnets to this framework?

1. Django integrates SQL into its container.

2. Django is a standalone server-solution that can be hosted on a remote or local server.

3. Firewall and reach, configuration of ports and sockets can be efficiently managed in Django.

4. Combined with the flexibility of Python, Django has added robust deployments.

5. Flexibility in creation of models and handlers.

6. Easily create apps and integrate several of those and manage them under one admin panel.

7. Django provides an inbuilt admin panel which saves a lot of code of course that can be customized.

That’s it. So when are you shifting to Django?
Complex Data Types in Python
By: Japneet Singh Chawla
Tags: Beginners Guide, Programming, Coding, Python

In the previous blog we talked about Simple Data Types in Python, if you want to read about the basic data types in python have a look at the blog: https://medium.com/analytics-vidhya/data-types-in-python-c23b8178f96d

Lets talk about some more data types in python:

LIST

List is basically a collection of elements of same or different data types.

Let’s discuss a use case of using lists.

You have to store marks data for the student: english, maths, science

We can either use 3 different variables for each student.

var eng=90 var maths=99 var sci=89

If we keep on adding these type of variables for every student we can have a large number of variable, creating a highly unmanageable code.

The second option is to make one variable and store all three marks in it like

var marks_1= [90,99,89]

var marks_2= [94,88,81]

var marks_3= [91,90,82]

This new data structure that we have used to store the data is called List.

It can have data of same or different types e.g

var student_1=[“The Data Singh”,”28”,”SDE”] var marks_1=[90,89,69]

List is one of the widely used data type in Python.

Some of the most used methods of list are:

marks_1.append(99) # to add a new element in a list marks_1.extend([10,20]) # to add all elements of a list to existing list marks_1.pop(index) #pops a value from the list just like a stack marks_1.reverse() #reverse. a list marks_1.insert(index, value) #add a value to a particular index

Now that we know how to create a list and add values to a list, let’s see how can we access elements from a list.

List Indexing

Indexes in list start with 0 in python.

In order to access elements from list you can use the following commands

marks_1[0] #will access first element of the list marks_1[1] # will access the second element of the list and so on… marks_1[-1] #will access the last element of the list marks_1[-2] #will access the second last element of the list and so on…

List Slicing

We can also access more than 1 elements in a single statement using list slicing.

marks_1[1:5] #this will access elements from 1 to 4 i.e 1,2,3,4. 5 is not included in the elements

We can also skip the start or the ending point in this slice

marks_1[:5] will give all elements till 5 i.e 0,1,2,3,4

marks_1[1:] will give all elements from 1 till the end

In the next post we will discuss the use case and use of the Dictionary Data type.

Do have a look at my channel on youtube : The Data Singh

My Blogger link: Tech Scouter

Happy Learning