fabiochiu/medium-articles huggingface dataset:
How to show a Dialog in Android Kotlin ?
Tags: Android App Development, AndroidDev, Androiddevweekly, Android

Now we will create an success alert dialog .

Workflow

We we create a fun succeseAlertDialog() in a .kt class file because if we have need same dialog more than one place in a screen. After we will set click event on a button and call the succeseAlertDialog() in it’s body. Just run the code.

Let’s coding .

MainActivity.kt

class MainActivity : AppCompatActivity() { override fun onCreate(savedInstanceState: Bundle?) {

super.onCreate(savedInstanceState) setContentView(R.layout.activity_main)btn_booking.setOnClickListener {showBookingAlertSuccessfully() }

}

private fun showBookingAlertSuccessfully(){

val bookingAlert = AlertDialog.Builder(this)

bookingAlert.setTitle("Booking Successfully.") bookingAlert.setMessage("Your Booking is completed.

Your Servicing Date is 2020–12–29

and Time is 15:58 .



For any help please visit on help page.")

bookingAlert.setIcon(R.drawable.type_success)bookingAlert.setPositiveButton("Ok"){dialog, id ->

val intent = Intent(this,MyServicesActivity::class.java)

startActivity(intent)

finish()

} val alertDialog = bookingAlert.create()

alertDialog.setCanceledOnTouchOutside(false)

alertDialog.show()

}

}

Note; if you have to use the dialog in fragment then you have to use activity (this) .
How to Create and Set up Firebase for Android
By: Golap Gunjan Barman
Tags: Android App Development, Android, AndroidDev, App Development, Firebase

In this tutorial, we’re going to see how to create a Firebase project and set up it with the android project. Before that, if you do not know what Firebase is and why we use firebase then go through my previous tutorial on Firebase.

All though, I give you a brief about firebase. “Firebase is a Backend-as-a-Service — BaaS — a platform developed by Google for creating mobile and web applications. Firebase is Google’s mobile application development platform that helps you build, improve, and grow your app.”

Want to know more, click here.

No, let’s see to create and set up Firebase for the android project

Part 1: Create the Firebase project

1. Go to the official page of Firebase (https://firebase.google.com/) and log in with your Google account. For Firebase you just need a Google account.

As you see here I logged in with my google account.

2. After logged in, click go to console to create a firebase project.

After clicking on the console, you see a button “Create a project”, click here to create a firebase project. After that, you need to give a name for your project and accept their terms and conditions.

Part 2: Setup Firebase for Android

3. After creating the Firebase project, you see an android button. Click here to setup the firebase for android.

4. First, you need to enter the android package name, which you will find in your applicationID in your app-level build.gradle file.

for SHA-1 certificate, go to your gradle button from right sidebar > app > android > signing report > SHA-1 certificate.

5. Then, download the google-services.json file and add to your app folder (android > app > google-services.json).

6. After that, add these code and dependencies in your android project (both in the app-level build.gradle file and project-level build.gradle file).

7. After adding the dependencies just click the NEXT button.

8. Now you can see your app in your Firebase console.

Just follow the steps and you will able to create a firebase project and set up with your android project.

Make sure you have a google account so that we can log into the firebase.

For more visit www.gbandroidblogs.com and show some love by clicking on the clap button.

Happy Coding!!
How to remove Android Application from the recent list
Tags: Android App Development, AndroidDev, Sofware Engineering, Technology, Remove App Recent List

There are two methods for removing the app from the recent list of app.

1. Using Manifest file

2. Programmatically

1) Manifest File

As we know Manifest file is used to declare all activities using the activity tag in XML. So if you want to remove the app from the recent list in Android then you are at the right place. We will discuss both methods in this article. In your activity tag, you have to mention three attributes which are listed below.

1. android:excludeFromRecents=”true”

2. android:taskAffinity=””

3. android:launchMode=”singleInstance”

2) Removing the app from the recent list programmatically in android

We can use finishAndRemoveTask method to remove the activity from the recent app list.
Rating Bar in Android Kotlin?
Tags: Android App Development, Androidappdevelopment Usa, Android, AndroidDev, Androiddevweekly

After Rating

RatingBar in Kotlin

Android RatingBar is a user interface widget which is used to get the rating from the customers or users. It is an extension of SeekBar and ProgressBar that shows star ratings and it allow users to give the rating by clicking on the stars.

In RatingBar, we can set the step size using android:stepSize and it will always return a rating value as floating point number such as 1.0, 2.0, 2.5 etc. By using, android:numStars attribute we can specify the number of stars in RatingBar. RatingBar is used to get ratings form users or customers about the product, movie or hotel experience etc.
Android RecyclerView With Kotlin Sealed Classes
By: Siva Ganesh Kantamani
Tags: Android, Mobile, AndroidDev, Programming, Kotlin

Create the Kotlin Sealed Class

The first thing we need to do in this approach is to create all the data classes we wish to use in the adapter and then link them in a sealed class. Let’s create a bunch of data classes:

These are few data classes that I wish to display on the list based on the data from servers. You can have as many data classes as you wish. This approach works at scale.

One of the best things about it is that we can maintain loading states, headers, footers, and more without writing any extra classes. You’ll know how to do it in a bit. The next step is to create the sealed classes that hold all the necessary data classes:

Sealed class with custom models

As I said before, we can add headers and footers without any extra work from the RecyclerView by using the object in Kotlin:

Sealed class with header and footer

Now, we’re done with the implementation of our sealed class.
Memoji — Welcome to Android
By: Muhammad Abrar
Tags: WhatsApp, Android Apps, AndroidDev, Emoji, Apps

Memoji is the name of Apple’s iPhone functionality that lets users create an emoji that looks like them. You can build your likeness from different categories like hairstyle, eyes, head shape, and so on. This becomes your Memoji. Now memoji is based on the iphone’s Animoji , that uses apple’s face id camera system to track your facial movements and translate those movements into the emoji that has your facial features (looks like you).

The customization level that memojis offer has taken the internet by storm. Tom Boellstorff said in his article that “memojis are the greatest invention of apple since iphone”.

Iphone 📱 has Memojis , Android has ?

Now for the user like me who want memojis but have never used iphone ,there was a lot of frustration invloved, but the great news is “where there is a will there is a way”. The creativity and freedom that android offers to its Genius Devs is unmatchable. Although Competing with memojis for iphone, google in introduced the mini emoji in their google keyboard app “G board”. but that lies far from class and quality the memojis have specially in the chat.

Now since memojis and emojis are mostly used in chats, and knowing the fact that whatsapp was the second most downloaded non-game app on the Google Play Store (again, after TikTok), but seventh on the Apple App Store, with More than 65 billion messages are sent via WhatsApp every day shows that how non trivial the role of memojis can be in our chats on android.

Memojis for Android

Being a fan of Memojis and Android developer, I have found a pretty decent way of getting genuine memoji from iphone and re use it as a sticker in whatsapp chat. Though the process takes quite an effort and time but i have now managed to make ,collect and catagorize this great memoji iphone collection into an android app with weekly updates.

Memoji Apple Stickers for Android — Sortscript (Playstore)

PROS : The App has high quality memojis of iphone with the first app in the play store to have proper categorization of memoji based on the hair color and face colors. The app started with 5 categories and has now grown to more than 15 categories with more than 300 stickers in a little time.

Cons : Although the app comes with great memoji collection however users usually look for creating their own memojis which unfortunately this memoji app doesn't offer for the time being.

Public Opinion :

Most Recent Reviews of Memoji Stickers App by Sort Script — Goolge Play Review Section

Although google has not come up with a feature that can beat memojis by apple but thanks to the individual efforts by android Developers , Android Users still can make their whatsapp chat more expressive and funny by using such cool memoji for android apps .
Android : SearchView Implementation
By: Anubhav Arora
Tags: Android App Development, Searchview, Android, AndroidDev, Search

Search View widget provides a search interface to the user to search a query and submit the request to the search provider and get a list of query suggestions or results.

In this article I will cover the implementation of the SearchView widget.

Let’s start with the first implementation

Create a menu file as follows:

<?xml version="1.0" encoding="utf-8"?>

<menu xmlns:android="http://schemas.android.com/apk/res/android"

xmlns:app="http://schemas.android.com/apk/res-auto">



<item

android:id="@+id/menu_search"

android:icon="@drawable/ic_search"

android:title="@string/search_menu"

app:actionViewClass="androidx.appcompat.widget.SearchView"

app:showAsAction="ifRoom" />



</menu>

Here the main thing to notice is the

app:actionViewClass="androidx.appcompat.widget.SearchView"

actionViewClass defines the class of a widget that implements an action, in our case it is the SearchView.

Our Activity or the Fragment should implement

SearchView.OnQueryTextListener

make sure you implement

androidx.appcompat.widget.SearchView and not

android.widget.SearchView

This interface will require us to override the following methods :

override fun onQueryTextSubmit(query: String?): Boolean {

TODO("Not yet implemented")

}



override fun onQueryTextChange(query: String?): Boolean {

TODO("Not yet implemented")

}

onQueryTextSubmit(query: String?): will be triggered when we enter the text and press enter in the search-view.

onQueryTextChange(query: String?): will be triggered whenever we start typing something in the search-view.

You can choose the appropriate method as per your requirement.

Then in our Activity or the Fragment override the onCreateOptionsMenu method and add the following logic.

override fun onCreateOptionsMenu(menu: Menu?): Boolean {

menuInflater.inflate(R.menu.main_activity_menu, menu)

val search = menu?.findItem(R.id.menu_search)

val searchView = search?.actionView as SearchView

searchView?.apply {

isSubmitButtonEnabled = true

setOnQueryTextListener(this@AcitvityOrFragmentReference)

}

}

val search = menu?.findItem(R.id.menu_search) : searches the menu item

val searchView = search?.actionView as SearchView : converts our search menu item into an actionview of type SearchView

searchView?.apply {

isSubmitButtonEnabled = true

setOnQueryTextListener(this@ListFragment)

}

In the above two lines SearchView widget is used to enable the submit button for non-empty search queries and set the listener.

And there you have successfully implemented the SearchView.
Infinite Lists With Paging 3 in Jetpack Compose
By: Vivek Singh
Tags: Android App Development, Android, AndroidDev, Paging Library, Jetpack Compose

Infinite Lists With Paging 3 in Jetpack Compose

Image on Jetpack Compose offficial docs

Previously there was no official support for paging in Jetpack Compose, but with the release of compose version 1.0.0-alpha06 , jetpack officially released the paging-compose library which provides integration between compose and Paging 3 library.

As a quick recap from my original article on how to use Paging 3 library to display data in a RecyclerView , paging is an efficient way to load small chunks of data at a time, and as soon as the user reach at the end of the list, more data will be loaded.

Code snippets in this article are taken from:
With App Doctor, you can scale your development capabilities without scaling the cost.
By: App Doctor
Tags: AndroidDev, App Maintenance, App Development, Custom App Development, iOS App Development

Looking to scale your development capabilities?

Don’t make this mistake!

We see it all the time. Companies that try to cut corners when they scale their web and app development and the results are disastrous. They hire a “friend-of-a-friend” or a solo freelancer who is unable to adjust and grow with their ever-changing and ever-evolving needs. They end up spending more than they should’ve because they have to do everything twice.

With App Doctor, you can scale your development capabilities without scaling the cost.

App Doctor eliminates this common situation once and for all. We can tackle any and all web and app development projects at a price that doesn’t break the bank.

Learn how we can support your development capabilities without scaling the cost!
Examining and Cleaning Data
By: Chijioke Godwin
Tags: Data Cleaning, Python, Data Science, Pandas, Jupyter Notebook

Examining and Cleaning Data

Image by Author

Data science beginners tend to struggle with Data Cleaning. They don’t know the right commands to use, they don’t understand why one command works with a column but returns error for another. Having to explain this over and over I decided to write a comprehensive guideline that I can always refer beginners to.

The first thing on getting a new dataset is to inspect, clean and transform into a format suitable for use. Datasets are cleaned to eliminate inconsistencies, errors, empty values and to avoid inaccurate, imprecise, meaningless and inconsistent results, insights and models. Before cleaning is performed, the dataset is inspected and scrutinized to know the right cleaning steps to perform.

Cleaning is a major part of the preprocessing operation and can be performed using Python (NumPy and Pandas) or Power BI (Power Query and M). This article is based on Python using Jupyter notebook to run commands and is focused on data cleaning only. Data cleaning is not straightforward, it is time consuming and due to the peculiarity of dataset, it is not easily automated. However, there are certain commands that are frequently used and we’ll be highlighting some of them in this article.

Most statisticians and analysts are comfortable with quantitative data but not qualitative data so I will briefly explain data classification and statistical levels of measurement. It is quite different from datatypes and data fields and even though most of our learning models will require the encoding of our data to numerical types, understanding the levels of measurements and data classification will help us understand these conversions and the operations that can be performed on them.
Fastest way to install Geopandas in jupyter notebook on Windows
By: Tanish Gupta
Tags: Folium, Data Science, Visualization, Geopandas, Jupyter Notebook

Ahmedabad (India) City Visualization made by me using Geopandas and Folium.

Hello everyone. Recently I was working on a data science group project in which my part was to make Geospatial visualizations which requires python library Geopandas to be installed in my system. I had two options either do it in jupyter notebook or Google Colab.

I chose jupyter notebook but faced a lot of problems while installing Geopandas.There were not a single good resource on internet which can help me to install Geopandas properly without any errors. So, I decided to write an article which will help all the data science enthusiasts in installing Geopandas easily.

In this article I am going to show the fastest and best way to install Geopandas in jupyter notebook on Windows.So, let’s get started.

First of all, if you will import geopandas without installing it, it will show this error which means we need to install it first.

Installing Geopandas

Follow the steps given below to install geopandas.

Note- Anaconda is assumed to be installed on your system.

Step 1- Open Anaconda command prompt and create new environment variable say “geo_env” in our case using the command given below.

conda create -n geo_env

Step 2- Activate this environment “geo_env” and add and set conda-forge channel.

conda activate geo_env conda config --env --add channels conda-forge conda config --env --set channel_priority strict

Step 3- Install Geopandas in the environment just created.

conda install geopandas

Verify that it is installed by running python in your terminal and then running following code.No errors means geopandas is succesfully installed.

import geopandas exit()

Step 4- Next task is to install jupyter notebook in this environment,as by default it is installed only in “base” environment.

conda install jupyter notebook

Step 5- Now add our environment to jupyter notenook.

python -m ipykernel install --name geo_env

That’s it ! Geopandas is succesfully installed in your environment variable. And now there are two ways to run it either through Anaconda Navigator or Anaconda command propmt.

Running Geopandas in Jupyter Notebook.

First way- Using Anaconda Navigator.

Open Anaconda Navigator,go to Environments section then select the environment,in our case geo_env. Then click on “play” button and select “Open with Jupyter Notebook”.

Second Way- Using Anaconda Command Prompt.

Follow the following steps.

Step-1 Open Anaconda command prompt.By default it is in “base” environment.

Step 2- Change it to the environment we created “geo_env” and installed geopandas by activating the environment.

conda activate geo_env

Step-3 Run “jupyter notebook” command and get going by either creating a new file or editing an existing ipynb file. Make sure you are working in the environment we have created.
Use nbdime to Visualize Your Jupyter Notebook Diff
Tags: Diff, Jupyter Notebook

Jupyter Notebook is our good friend on the data science work. We usually leverage git to version control the *.ipynb file, but when you try to get the chances between different versions, we meet some problems.

The *.ipynb is a structured file that contains all the information for the page, but we usually only concentrate on the code changes. Diff on the *ipynb file directly will not give us a good one, so here comes the nbdime tool.

The nbdime can be installed via pip, and besides the command-line tool, it provides a web-based tool to visualize your diff.

Install

pip install — upgrade nbdime

Apparently, this tool can be installed on win/mac/linux system. Since all my Linux servers do not have the graphical module, I prefer to install this tool on my windows machine and show the diff in the browser.

In the windows system, we can find some useful commands after the nbdime installation.

Under C:\Users\username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Scripts

Add this path to the system env and start a web-based diff tool.
FT232H with HT16K33
Tags: Rust, Arduino, Jupyter Notebook, Adafruit

FT232H with HT16K33

HT16K33 (datasheet link) is very similar to TM1637. It is also widely used in display modules such as Adafruit 8x8 LED matrix (schematic link and LED hardware link), or this 4-letter pHAT module (link).

Again Adafruit provides nice Arduino/Python language support (link) for its modules and HT16K33. But we can certainly experiment on HT16K33 with a FT232H module as well.

Comparing with TM1637, in terms of its I2C interface and digital commands, HT16K33 has the two main differences: the opcodes and that its I2C byte order is MSB first. So the following Rust Jupyter notebook is almost the same as the TM1637 one.

One more thing, if you are driving pHAT with a FT232H module, make sure to connect USB +5V to pHAT as well (pin 2 on the header). The alphanumeric letter bit patterns can be found in the python file (link). Feel free to experiment and make better looking letters in some cases.

The next step is to implement a maze game like this (link) in a Rust Jupyter notebook.
Check my data science portfolio …still on the making but getting ready …. https://github.com/panditadata/Another-WorldBank-With-A-Little-Bit-Of-Spice/
By: Panditadata Panditadata
Tags: Data Science, Jupyter Notebook, Data Science Portfolio, Statistics

panditadata/Another-WorldBank-With-A-Little-Bit-Of-Spice

Coursera Project to calculate basic statistics of a World Bank Dataset The indicator selected to do all the stats…
Les raccourcis clavier sur Jupyter Notebook!
By: Joséphine Picot
Tags: Shortcut Keys, Python, Raccourcis Clavier, Notebook, Jupyter Notebook

Hi! I am a last year student in an engineering school and I am passionate about data science. Whatever the type of data, I love to dig in!
Exporting and Visualizing Data using python and Jira REST API
Tags: Jira, Jupyter Notebook, Python

Jira is a issue tracking software designed and developed by Atlassian that allows bug tracking and agile project management. Jira is widely used in agile environment due to its well-managed workflow mapping and issue tracking ability.

In this article we will export data from Jira using Python and will visualize data using Matplotlib, a comprehensive library for creating static animated, and interactive visualization in Python.

We will start with installing dependent libraries in our Jupitar notebook:

!pip install jira

!pip install matplotlib

!pip install python-csv

!pip install pandas

Then we will import the libraries:

from jira import JIRA

import csv

import pandas as pd

import matplotlib.pyplot as plt

Connect to Jira instance

Follow this link for generating API Token: https://confluence.atlassian.com/cloud/api-tokens-938839638.html

For this tutorial we will search all the issues assigned to current user and create a bar chart showing how many issues assigned to the current user each month. In order to get all the assigned issue we will leverage JQL ( JIRA Query Language).

issues = jira.search_issues('assignee = currentUser()', maxResults=500)

Following code will iterate through all the issues and will write the issue created date in “text.csv” file. We are using slice function to take only year and month.

slice_object = slice(7)

for issue in issues:

with open('text.csv', 'w', newline='', encoding='utf-8') as csvfile:

fieldnames = ['date']

writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

writer.writeheader()

for issue in issues:

writer.writerow({'date': issue.fields.created[slice_object]})

Now we will read “text.csv” file using pandas

df = pd.read_csv('text.csv', sep =";")

Use the pandas head function for quickly testing if our object has the right type of data in it

df.head()

Now we will count all the unique dates and sort them

count_unique = pd.DataFrame(df.date.value_counts().reset_index().values, columns=["Date", "Count"])

count_uniqueindex = count_unique.sort_index(axis = 0, ascending=True)

count_uniqueindex

df = count_uniqueindex.sort_values(by=['Date'])

Finally we will visualize our data using Matplotlib

fig = plt.figure()

ax = fig.add_axes([0,0,1,1])

Date = df['Date']

Count = df['Count']

ax.bar(Date,Count)

plt.xticks(rotation=45, ha='right')

plt.show()

Full code file: https://github.com/no9600hp/JiraVisualization/blob/master/JiraVisualization.ipynb
Create a conda environment to run JavaScript with Jupyter notebook
Tags: Programming, Jupyter Notebook, Conda, JavaScript

Photo by Joshua Reddekopp on Unsplash

Jupyter notebook offers an interactive way to execute your code fragments and see results immediately. You can turn your toy codes and sample results into presentable documentation with no sweat. It has become a staple in early-stage ideation and proof-of-concept for Data Scientists. In addition to its predominant use-case with Python kernels, Jupyter also offers JavaScript kernel to help you achieve similar things.

Through my process of installing the JavaScript kernel for Jupyter, I encountered some incompatibility issues that caused trouble starting the kernel. This article shows a working approach on how to install JavaScript kernel for Jupyter notebook on macOS. Hope it can help you so you don’t need to waste too much time as I did.

Before we get started, you should make sure conda is installed on your macOS. If it has not been installed, you can find instructions here. Once it’s installed, you can follow the steps below to create a conda environment with Jupyter notebook that has the JavaScript kernel installed, which runs with node.js.

Install Homebrew and install ZeroMQ using homebrew.

ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew install pkg-config node zeromq

Create a new conda environment with Python3.6 and activate it. We’ll call it jupyter-js-kernel for convenience.

conda create --name jupyter-js-kernel python=3.6

conda activate jupyter-js-kernel

Install pyzmq and jupyter.

pip install --upgrade pyzmq jupyter

Install node.js for your conda environment.

conda install nodejs

Install JavaScript kernel for Jupyter with npm.

npm install -g ijavascript

Run the kernel installer script to finish the setup.

ijavascript --spec-path=full

In case ijavascript isn’t recognized, you can find the path to the installer script and run it directly. The path typically looks like the following

/Users/[User]/lib/node_modules/ijavascript/bin/ijsinstall.js --spec-path=full

Now you are all set. Try running your Jupyter notebook and see if you can create a new notebook with JavaScript kernel successfully. Let me know if you run into any problems in the comment section. Have fun!
Survey — JupyterLab and beyond
By: Luciano Resende
Tags: Jupyter Notebook, Jupyterlab, Surveys

The 2020 end-user survey is live!

https://www.surveymonkey.com/r/LCB7GBF

Survey Goals

The purpose of this survey is to gather information from the community in order to enable data-driven decision making about the future of JupyterLab.

The content of the survey focuses on the following categories:

Use Cases — who are they, what are their usage patterns, and what are they doing?

Tooling — what type of data & tools are they using for data science, and what are their pain points in doing so?

By participating in this survey, end-users will guide the roadmap of JupyterLab toward their use cases. Even if you don’t use Jupyter, your input is welcome (see initial questions).

Please share with your teams and friends.

Thank you.
Effortlessly Execute JupyterLab Cells in the Cloud
By: Jagane Sundar
Tags: Jupyterlab, Mlflow, Machine Learning, Artificial Intelligence, Jupyter Notebook

There are many occasions when Python code in a JupyterLab notebook may need to be run in the cloud. InfinStor offers the easiest way for cloud execution of Python code authored in JupyterLab.

InfinStor Starter Functionality

InfinStor Starter captures Python code from a JupyterLab cell, combines it with an execution environment, such as Conda or Docker, and saves it in the cloud as an InfinStor Transform.

Subsequently, InfinStor transforms can be run in a Cloud instance of your choice. InfinStor can also help you choose the cheapest cloud.

Additionally, InfinStor transforms may be scheduled for periodic runs in the cloud. Periodic runs can be scheduled hourly, weekly, monthly, etc. There is no need to create Airflow DAGs or configure any other job scheduler for this functionality — it is available straight from the InfinStor JupyterLab sidebar GUI.

Finally, all such transform executions are recorded in the included InfinStor MLflow service.

All of the functionality of InfinStor is accessed through the primary InfinStor GUI, which is the InfinStor JupyterLab sidebar. There is no need to use the command line interface, or to author xml/json/yaml configuration files.

Capture JupyterLab Cell and Save as InfinStor Transform

The first step in remotely executing python code from a cell in your JupyterLab notebook is to capture it, along with its execution environment, and save it in your InfinStor account as an InfinStor Transform

JupyterLab Cell Python Code

The python code in your JupyterLab cell cannot read data from the local file system. This transform may execute in a variety of environments — in an IPython kernel in your JupyterLab server machine, in a single VM in the cloud, or in a cluster of machines in the cloud. This requires that the input data to this code should be stored somewhere in the cloud, preferably in a Cloud Object Store

Capture Transform

In the following screen capture example, the cell contains a single line of Python code

print(‘hello world’ )

Select this JupyterLab cell and press the ‘Capture’ button in ‘Transforms->Develop’
Literate Programming using Jupyter Notebook
Tags: Python, Fastai, Data Science, Data, Jupyter Notebook

Overview

As developer, I use vim as my code editor for my personal projects, and Visual Studio Code at work. Occasionally I use Emacs as I really like org mode, especially using org babel for reproducible research and literal programming, providing a computing environment for authoring mixed natural and computer language documents.

For those who are not familiar with literate programming, from Wikipedia

For data science projects, I use Jupyter notebook for exploratory data analysis, data preparation, wrangling, training and testing the models and eventually coming out with the final model for production. During this process, sometimes the code I wrote in the notebooks need to be extracted and packaged as a python module or library for further reuse.

I have been using fast.ai library in few of my projects and it is a cool library which make machine learning accessible and easy for everyone to get started. fast.ai library is developed through a notebook approach using nbdev. After watching a recent video by Jeremy I decided to try this approach.

Python Programming using nbdev

nbdev is a library that allows you to develop a python library in Jupyter notebooks, putting all your code, tests and documentation in one place. There are detailed instructions and tutorials provided at the website so I will not repeat the details here.
Manipulando grandes datasets com AWS EMR
By: João Lucas Flauzino Cassiano
Tags: AWS, Pyspark, Jupyter Notebook, Aws Emr

Learn more. Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Make Medium yours. Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore
Getting started with JupyterLab, Pandas, NumPy, and Matplotlib
Tags: Python, Data Science, Data, Virtual Environment, Jupyter Notebook

Setting up your Env:

There are a few steps to get to this part, namely setting up your virtual environment. Let’s go over an environment (envs) in this brief description: envs are used for package and library management, which is another way to say, this will help you put all the right tools for this project in one place.

I’ll be using Conda envs (docs). First, go to the docs, install Anaconda given your respective OS. Open spotlight search on Mac with the shortcut command + space , then type terminal then enter . That should open your terminal, which will look something like this:

Notice the (base) you can see here in mine? That is your root conda environment. If you want a more in depth guide to conda, read this article written by Matthew Sarmiento. If you don’t see (base) , you can activate it by typing:

conda activate

If that still doesn’t work, it means the installation was not correctly done; here is where you need to go read the docs a little more in depth 🤷‍♂️.

Now, create a conda env for the specific project with the following code (change project_name to your actual project’s name):

conda create --name project_name python=3.8

Also, run the following code to install a kernel (make sure to pip install ipykernel first ):

python -m ipykernel install --name project_name_kernel --display-name "project_name_kernel"

Python Kernels allows is essentially the backend execution for Jupyter. If you want a short vid on this, go here, or see the docs.

Notice here how it now says (project2) instead of (base) . That means you’re now in your project env. To see your envs you can use the conda env list command.

Installations:

There are a couple different ways to install your dependencies, I’ll be covering one.

Right, dependencies, what are they? Well, basically the tools you’ll be using for your project; I’ll focus on Pandas and NumPy here, but keep in mind there are a lot of dependencies and libraries for many different things.

Now, let’s install Pandas, NumPy and Matplotlib using the following commands:

conda install -c conda-forge pandas

conda install -c conda-forge numpy

conda install -c conda-forge matplotlib

conda install -c conda-forge jupyterlab

You’re ready to open up a Jupyter notebook, simply type in the following command:

jupyter lab --ContentsManager.allow_hidden=True

You can also simply use jupyter lab as the command, but I like to have access to all of the files in directory. Your browser should open a window like the following:

It should open in from an address similar to this localhost:8888/ . If you want to know more about localhost go here.

Now, start with your first notebook through the File tab; upon opening a notebook you’ll be prompted to choose a kernel. Choose the kernel created for this project:

Okay, now you’re ready to start your data analysis. I’ll add a second part that goes into the data analysis in the next few days.

GitHub | LinkIn | Portfolio | Twitter
Numpy Crash Course (Part 2)
By: Faiza Anan Noor
Tags: Python, Numpy, Matrix, Jupyter Notebook, Arrays

Stacking can also be done along the depth/height using dstack() function like in the above example.

SPLITTING ARRAYS:

Here the splitting is done using the array_split function which takes in the array we want to split, and the number of splits we’ll do.

Splitting along an axis:

In the first example, we split the array along its row.

In the second example, we split it along its column.

Splitting using vertically, horizontally using designated functions:

hsplit can be used to split the arrays horizontally

vsplit can be used to split arrays vertically

ITERATION OF ARRAYS:

We can iterate through arrays using simple for loops.

But for getting the scalar elements we need multiple for loops.

In the above example, we needed two for loops to loop through the 2D matrices arr1, arr3.

The outer for loop iterated through each dimension of the 2D array.

The inner for loop iterated through the elements of a particular dimension.

Here we iterated through a 3D array using 3 for loops.

Alternative to using for loops in this case:

Here the function nditer eradicates the hassle of creating multiple for loops to iterate through the scalar elements of higher dimension arrays. We can just use one for loop to serve the purpose by using this function.

ENUMERATION/GETTING INDEX NUMBER,VALUE:
Run Jupyter Lab on a Remote Server
Tags: Ssh Tunnel, Jupyter Notebook, Jupyterlab, Ssh

Jupyter lab is a great web application for Python coding. This post instructs you how to run jupyter lab on a remote server using an ssh tunnel. I assume you know jupyter lab and already used it locally. Now, you want to run jupyter lab on a remote ubuntu server and access this jupyter lab server from your local machine (e.g. notebook). This way you can code directly on a remote server (e.g. a DGX machine) trough the web browser of your personal end device. — Hey, that’s cool.

Turn on your laptop. Turn on the remote server. Enable an ssh connection between your local machine and the remote server. Establish an ssh connection and set up an ssh tunnel. For instance:

$ ssh mercury@192.168.127.123 -L 8889:localhost:8889

opens an ssh tunnel on port 8889 and logs in the user mercury on the server 192.168.127.123.

3. Now, you are logged in on the remote server. Install conda, jupyter lab on the remote server if not already done. I am doing this in the snippet below and also create the conda environment named jlabenv.

mercury@DGX:~$ sudo apt install conda

(base) mercury@DGX:~$ conda create --name=jlabenv

(base) mercury@DGX:~$ conda activate jlabenv

(jlabenv) mercury@DGX:~$ conda install jupyterlab

4. Start jupyter lab without opening a browser and let it connect on the port you set the ssh tunnel on. Below, I type a command to start jupyter lab as a background process using nohup and dump the output to a specified logfile:

$ nohup jupyter lab --no-browser --port=8889 > ./jupyterlab.log &

The &-sign allows me to get a new prompt.

5. Get the token to connect to the jupyter lab server. You can copy the token from the log file. Just open it with your favourite editor. I choose vim:

$ vim ~/tmp/jupyterlab.log

6. Copy and paste the given URL to access the jupyter server from your local machine.

Here you go!

Just make sure not to close the ssh tunnel. Otherwise you’d loose the connection to the server. If you lose the connection, don’t worry, you can just reconnect and return to the jupyter lab server where you left.
Plotting Matrix using Python
By: Shashi Vishwakarma
Tags: Python, Beginners Guide, Matrix, Machine Learning, Jupyter Notebook

I have recently started to sharpen my machine learning skills and exploring couple of new interesting problems. In my college, I was quite good in mathematics but really haven’t seen so far its implementation but machine learning is place where I find out its true use case.

Let me start with something very basic and going forward in couple of blogs share my experience of machine learning — Shift from Data Engineering to Machine Leaning Engineering.

Today I used Python and Matlab API to plot a 2-D matrix. Its quite exciting and good to see something colorful. Specially for a person like me who has always spend his most of time in back-white screen(linux terminal).

Pre-requisite : I have installed Jupyter notebook (most popular in data scientist) using Anaconda.

Now I am going to create a matrix using numpy followed by matlab to plot.

Source Code :

import numpy as np

import matplotlib.pyplot as matrixPlot



#Create a matrix

A=np.matrix('1,2,3;9,4,7;-1,9,6')



#plot matrix

matrixPlot.ion() #Turns interactive mode on

matrixPlot.imshow(A, interpolation='none')

pyplot() function take multiple properties and one of them is interpolation property. You can explore by changing it someother values to checkout magic.

Lets set interpolation to ‘kaiser’

import numpy as np

import matplotlib.pyplot as matrixPlot



#Create a matrix

A=np.matrix('1,2,3;9,4,7;-1,9,6')



#plot matrix

matrixPlot.ion() #Turns interactive mode on

matrixPlot.imshow(A, interpolation='kaiser')

Reference : https://matplotlib.org/api/pyplot_api.html

Happy Coding..!!!
Beyond Jupyter Notebooks
By: Greg Jan
Tags: Data Science, Docker, Flask, Machine Learning, Jupyter Notebook

So all we need now is a Dockerfile to configure our container (given that Docker is installed). You can place the following file on the root of your project folder:

The base image used here is python 3.7-slim (which is Debian based). All the RUN commands update the base image with the latest packages. Two critical lines in the Dockerfile are to copy and install all the python libraries from the requirements.txt file. This means that the same library versions will be used on the container as on your local anaconda environment. To create this file run in a terminal (where your anaconda environment must be activated):

pip freeze > requirements.txt

The COPY line will then copy all the files and folders to the container. Finally the CMD line will launch the web app by running the app.py job to URL 0.0.0.0 and port 8080. This URL is a “all interfaces” address which will be accessible locally by using the localhost.

Don’t forget before building our container to update app.py by un-commenting the line 21 in order to route the app to URL 0.0.0.0 through port 8080 (and comment line 19).

Ok now we are all set. Let’s make that container:

docker build --tag <containername> .

It may take a bit of time depending on the speed of your connection. But once it is done you can launch the container:

docker run -p 8080:8080 <containername>:latest

Next step, we test. As in our local version we can run the following notebook, only updating the port from 5000 to 8080:

We get the same response as we got locally. You have made it. CONGRATULATIONS!

Now we could register this container to any Cloud vendors such as GCP, Azure or AWS and make it accessible to the world. But I will not cover it in this article.
VSCode Jupyter Notebook Keyboard ShortCuts
Tags: Shortcut Keys, Keyboard Shortcuts, Vscode, Shortcuts, Jupyter Notebook

I organized the keyboard shortcuts in VScode when using Jupyter Notebook.

I referred to VScode documentation.(https://code.visualstudio.com/docs/python/jupyter-support)

Ctrl+S : Save

: Save ESC : Change the cell mode

: Change the cell mode A : Add a cell above

: Add a cell above B : Add a cell below

: Add a cell below J or down arrow key : Change a cell to below

: Change a cell to below K or up arrow key : Change a cell to above

: Change a cell to above Ctrl+Enter : Run the currently selected cell

: Run the currently selected cell Shift+Enter : Run the currently selected cell and insert a new cell immediately below (focus moves to new cell)

: Run the currently selected cell and insert a new cell immediately below (focus moves to new cell) Alt+Enter : Run the currently selected cell and insert a new cell immediately below (focus remains on current cell)

: Run the currently selected cell and insert a new cell immediately below (focus remains on current cell) dd : Delete a selected cell

: Delete a selected cell z : Undo the last change

: Undo the last change M : switch the cell type to Markdown

: switch the cell type to Markdown Y : switch the cell type to code

: switch the cell type to code L : Enable/Disable line numbers

Please let me know if there is something that has to be changed.
How to automate scheduled running of a Python Notebook
By: Kundan Kumar
Tags: Task Scheduler, Automation, Jupyter Notebook, Python

Here is the process to automate or schedule running of Python script or a Jupyter notebook using Task Scheduler

Task scheduler is for windows only, for linux and Mac you can try https://snapcraft.io/mkcron and https://apple.stackexchange.com/questions/4714/os-x-is-there-a-built-in-scheduler-program.

This will work for both python script and a Jupyter notebook.

Step 1

In case of Jupyter notebook first convert it into a python script by going to file then download as python file.

Download a Jupyter notebook as a python file

Step 2

Now open Task scheduler and click on Task Scheduler Library.

Then on the right menu click on create task.

2. Then give the task a name as description

It will look something like this

create task dialog box

Step 2

You will need to have certain locations before moving forward

Location of installation of python interpreter : type where python in cmd to get the location Location of the python script

location of interpretor

location of the python script also note the name of the script

Step 3

Goto Action section of the task scheduler, here we will tell the task scheduler what to do, click on new

Inside the Program/script box paste the location of python interpreter Inside Start in: add the location where the python file is And finally inside the Add Arguments box paste the name of the python file press ok

Creating Action for the scheduler

Step 4

Now go to the Triggers section and click new, we will set the schedule for which the program should run

Select Weekly or daily or monthy depending on your requiments Select the time when the script should start optional set the expiration date

In my case I chose the script to run on every Monday, Tuesday, Wednesday and Friday at 8 hours, with no expiration date

creating trigger for the script

Finally

The new task will be created and status will tell if its ready or not, and the next run time will shown.

Scheduled Task created

To make your task a bit more better you can add a code to show notification at the end of execution.

task running along with notification

You can also add several print statements inside the python script file to see your progress when the script is executed.
Getting Started with Anaconda | ML-2
By: Elif Işıkhan
Tags: Python, Jupyterlab, Machine Learning, Anaconda, Jupyter Notebook

Hello Friends! In this article I will demonstrate to you how to create a new environment and the first example which is ‘Hello World’ in Jupyter Lab

Before all lets take a look ,

What is Anaconda Navigator?

Anaconda Navigator is a desktop graphical user interface (GUI) included in Anaconda® distribution that allows you to launch applications and easily manage conda packages, environments, and channels without using command-line commands. Navigator can search for packages on Anaconda Cloud or in a local Anaconda Repository. It is available for Windows, macOS, and Linux.

So let’s start….

Step 1 Installing Anaconda

Start with the Anaconda Navigator download first.

Go to the Anaconda’s official page

Select and download the one that fits your computer from the download options at the bottom of the page.

Check the second option for Windows download as below

Note : Check the box ‘Add Anaconda3 to my PATH environment variable’ if you want to use Anaconda in your Command prompt, Git Bash etc.

Step 2 Create New Environment

Now Launch Anaconda Navigator > Environments

The new environment will take less than 1 minute to occur (depending on the machine) and will be added to environment list.

You can also see many options in the right part and you can install & update packages, libs etc.

Step 3 Installing Applications

Go back to Home section and make sure the created new environment is selected at the top of the page and then Install application what you will use in your environment. I am going to install JupyterLab and Jupyter Notebook for now.

Step 4 Launch JupyterLab & Hello World

When you click Launch on JupyterLab you get this page on your localhost. (I jumped the file which I have created as Machine Learning)

As you see in Launcher tab,you can launch either a notebook and Python 3 or console in Python 3 and a few other things as well.

Click Notebook to create Hello World file.

There’s a few types of cells here. In this case we want to markdown because it gives us the rich formatted text the hash here. What you have here is that the details of the in the code so get run and you get this beautiful rich text format .

I can either restart kernel and restart clear or I can actually these clear outputs and or the output goes away.

I’ve illustrated to you so far is code and markdown and I’ve shown you that you can see when it is a code you can use it to calculate to perform computation print screen as well as storing the data into their variable in the RAM itself.

I hope that you find this article useful. Thanks for reading :)