fabiochiu/medium-articles huggingface dataset:
New Teacher Features | December 2020
By: Arienne Stevenson
Tags: Reading, Education, Literacy, Teaching And Learning, Edtech

⚙️ Customize settings

Customize your Simbi experience by setting your avatar, title, name, country and accent. Flip through the tabs to move into email notification settings, to change your email or password, and to customize your Simbi experience.

📋 Hear TellMes right from ListenIn

Educators can now listen to a student’s TellMe right from the ListenIn page of a story. This tool enables you to quickly listen to their thinking and understanding without having to return to the student’s activity page.

👨‍👧 Add existing students to groups

Groups are a fantastic tool for differentiated reading instruction. Now there’s a simpler way to add or move students between groups if their reading levels change or you’re ready to switch up literature circles! Click on Add Students, and check out the new option to add existing students!

🗺 New simplified student Journey

We’ve updated the student Journey to reflect student reading, and to keep them moving forward in their reading progress! No more, no less.

Ready to see these features in action?

Happy Reading! 📚🧡

Follow us on Instagram, Facebook, & Twitter to stay up to date on feature releases, giveaways, and more fun!

Educators! Join our Simbi Educator Community for lessons & resources.
A guide to using Tele-calling for Edtech
Tags: Sales, Tele Caller, Education, Edtech, Telecalling

2020 has been a major catalyst in enhancing the learning process not just for students and educators, but also for investors and edtech startups. Edtech companies saw their long-term business goals achieved within six months.

The pandemic has accelerated the adoption of digital technology in education with global edtech investment on track to grow over 15% in 2020. Edtech startups are helping schools and universities to adapt and evolve. Across the three main education categories K-12, post-secondary, and corporate training, technology is infused throughout the life of a learner. That throws up a lot of new opportunities in the B2C and B2B space.

Education requires a huge amount of human interaction and the tech in education should be used to improve efficiency. With Futwork, companies are doing just that. The tele-calling services offered by Futwork helps businesses scale their sales funnel in a cost-effective manner while saving the time of their in-house salesforce.

Edtech companies like uFaber, toppr, Vedantu have been working with Futwork on various use cases like lead validation, customer feedback, payment reminders, and booking demo sessions.

With companies growing at a breakneck pace and establishing a presence across India they often find it challenging to communicate with their user base in the regional language, given the diversity. With Futwork’s tele-calling companies are able to overcome that obstacle as we provide services in more than 10 languages based on the need of the business.

With tele-calling, edtech companies have been able to improve efficiency and reduce friction at each stage. Here is how we add value at each stage.

Lead Qualification: Our team of trained tele-callers ensures that you are only reaching out to the hot leads by qualifying the leads based on set parameters and your in-house sales team can focus on maximum conversions.

2. Demo Bookings: Edtech startups choose the freemium model to entice parents/teachers to their solutions. The tele-callers provide detailed information to the prospects ensuring they understand the product well and get them to attend a demo.

3. Sale closure: Tele-callers work on follow-ups and ensure the prospects complete the formalities and make the payment to get started, so your team can just focus on converting the leads.

4. Nurture: Timely feedback calls to users for user experience. Asking users for referrals and new leads ensuring the in-house sales team has maximum leads in the funnel throughout.

Top benefits of using Futwork for tele-calling to support your inhouse team:

Pay for output Zero fixed costs Multi-lingual callers On-demand tele-calling End-to-end project management

Optimize your sales funnel with Futwork, reach out to us at hello@futwork.com or fill in the form here and we will get in touch with you.
Telia and Clanbeat collaboration: Estonian and Russian languages coming soon
Tags: Students, Edtech, Clanbeat, Estonia, Wellbeing

‘So, when will Clanbeat be available in our native language?’. This is a question we’ve heard a fair few times in 2020 when chatting with teachers in our home county.

Well, Estonian educators, your wish has been graciously granted. We’re thrilled to announce that with the kind support of Telia, our app will be available to use in both Estonian and Russian from the beginning of 2021.

Earlier this year, Telia took a closer look at how children and young people are managing distanced learning in the Baltic and Nordic countries. Their research exposed both the joys and pain-points students are facing today.

Photo by Julia M Cameron from Pexels

Born from Telia's research findings and our shared motivation to create a positive impact, our collaboration was established. We’ve since been engaged in a joint mission to support student mental health and personal development, ensuring potential issues are identified and overcome at the earliest possible chance. By localising languages, teachers will be able to better engage their students, helping them to flourish in both their life and learning.

Elo Võrk, Head of sponsorship and CSR at Telia Eesti said: “For a very long time, we’ve been looking for a digital solution that would help prevent bullying at schools and support younger people’s mental health. When we heard of Clanbeat via the Education and Youth Authority, we realised immediately that this was just what we’d been searching for.

“What’s especially important for us is that the solution is created together with its main users — young people. We really hope that through Clanbeat another very cool innovation in education will hail from Estonia, sharing our positive experience with the world.”

At Clanbeat HQ we’ve warmly welcomed the help of one of Telia’s own experienced developers, as well as assistance with translations. Telia’s contribution has been pivotal in enabling us to undertake and sustain such a desirable feature for our local teachers.

From the entire Clanbeat clan, we would like to express the deepest gratitude to the Education and Youth Authority team for igniting this cooperation. And of course, a huge thank you to Telia for your incredible support.

Watch this space for more updates from us shortly.
A New Era for Camp Supernow
By: Lyndsey Wheeler
Tags: Parenting, Virtual Learning, Edtech, Education

A New Era for Camp Supernow

When we first launched Camp Supernow, it was the dawn of our first ever summer in quarantine. Moved by the stories of dozens of friends and colleagues who were struggling to juggle working from home, educating their kids, and keeping everyone sane and happy, we set out to create a meaningful experience that made the lives of families easier.

As a band of experience designers and community builders, our goal was to create a special place where kids could escape from the stress of the pandemic, socialize and have fun together, and give parents a break from their duties for a few hours each day. On June 1, our virtual summer camp was born.

Over the summer, our belief in the power of community to support kids (and parents) only grew. We built a culture of boundless spirit, gratitude and joy, that kept kids coming back to co-create and bond with peers from all over the world. We realized it is the social and emotional learning — delivered via magical counselors, zany themes and joyful activities — that makes Camp Supernow so special.

Though the future is unknown, we do know this — kids’ ability to connect with others, cope with adversity, communicate their feelings and maintain self-confidence are the ultimate indicators of future success and happiness. As we move into 2021, we are reaffirming our mission to use the power of community to cultivate critical social and emotional skills in kids and support parents on their parenting journey.

Now to the exciting part. With a reinvigorated focus comes a fresh new look. We are thrilled to unveil the new Camp Supernow website and an entirely new look and feel to go along with it. The new brand was designed to bring to life the creativity, energy, joy and connection that defines the Camp Supernow experience.

We’ve got even bigger things cooking for 2021, but a few other updates until then..

Our Board of Super Experts — As we double down on our commitment to foster social and emotional learning in kids, we’ve brought together a board of Super Experts to serve as content and experience advisors across parenting, child psychology and social and emotional learning. Meet our experts.

New Super Parent Resources — The Supernow community doesn’t end with kids. We’ve heard from our Superfam and are excited to roll out new parenting resources so no one has to go it alone. Look out for monthly parenting workshops, newsletters, and community spaces and more. Next up, we’re hosting a workshop on Navigating Pandemic anxiety with Kids. Sign Up Here.

We’re so grateful for our beautiful community of families, friends and advisors who continue to help shape what Supernow is becoming. To more magic in 2021!
Из фармацевтики в продажи
Tags: Education Technology, Edtech Tools, Education, Edtech Startup, Edtech

Learn more. Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Make Medium yours. Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore
What is Edtech? +4 Ways AI Is Shaping Education
By: Natalie Koltunova
Tags: Trends, Tech Trends, Edtech Tools, AI, Edtech

Edtech is a combination of modern technologies that helps to switch from the familiar studying process to an online or even personalized learning experience. Edtech includes all types of software and technological tools that influence the educational process at every level.

Let’s dig a little deeper into this question. EdTech is constantly pushing teachers and students towards new horizons. Here are four benefits of using educational technology for students of all ages, as well as four benefits specifically for teachers.

Check 4 reasons why students benefit from edtech below:

Round-the-clock learning

Educational technologies allow students to learn from anywhere and everywhere. All written materials and video content are available 24/7 and access to learning has never been easier for students. They can use their tablets, smartphones, and laptops wherever they are, whether they are on their way home or on holidays. The only thing that’s needed is a connection to the internet. The ability to study at any time and from any location motivates students to learn as it is no longer strictly necessary for them to be physically present in the classroom.

Read the full version: https://bit.ly/33PWWON
Coding with PyTorch in Android Phone (Termux)
By: Chulayuth Asawaroengchai
Tags: Android, Deep Learning, Pytorch, Termux, Machine Learning

I previously wrote a blog about how to build and install Tensorflow in Android Termux environment here:

https://medium.com/analytics-vidhya/developing-tensorflow-on-android-phone-cfc4297b676e

This time, it is PyTorch turn. Luckily, PyTorch setting up process is fairly straight forward compared to Tensorflow. There is no complex build system like JVM/Bazel. All you need are cmake, along with some modification in ARM assembly code, then you are good to go.

I. Prerequisites

Note that if you already followed my last blog about installing Tensorflow inside Termux, you can skip this prerequisite section.

Install Termux from Google PlayStore. Launch Termux. Make sure Termux is up to dated: pkg upgrade Install wget and proot: pkg install wget proot Create directory for installing ubuntu: mkdir ubuntu && cd ubuntu Download ubuntu chroot installation script: wget https://raw.githubusercontent.com/Neo-Oli/termux-ubuntu/master/ubuntu.sh Run the installation script: bash ubuntu.sh

Now, we can start ubuntu by running the script: ./start-ubuntu.sh

Next, we will check and install updates on Ubuntu image.

apt-get update && apt-get upgrade

Install some prerequisites.

apt-get install git wget vim zip build-essential python python3 python3-dev python3-pip libhdf5-dev

II. Setting up PyTorch inside Termux environment.

Clone PyTorch from GitHub.

git clone --recursive https://github.com/pytorch/pytorch

2. Checkout the last known version that can be installed on ARM architecture.

cd pytorch

git checkout v1.5.0

3. Modify a part of ARM assembly code that is not originally compatible with ARM CPU in Android systems.

The description for the modification is to change how the operands access register values from 4 shorts into 16 bytes (which are exactly the same meaning in ARM CPU architecture). This change is necessary because ARM NEON assembler in Android does not understand 4s data type.

File: aten/src/ATen/native/quantized/cpu/qnnpack/src/q8gemm/8x8-dq-aarch64-neon.S

Line: 662 Change from: MOV V8.4s, V9.4s

MOV V10.4s, V11.4s

MOV V12.4s, V13.4s

MOV V14.4s, V15.4s

MOV V16.4s, V17.4s

MOV V18.4s, V19.4s

MOV V20.4s, V21.4s

MOV V22.4s, V23.4s into: MOV V8.16b, V9.16b

MOV V10.16b, V11.16b

MOV V12.16b, V13.16b

MOV V14.16b, V15.16b

MOV V16.16b, V17.16b

MOV V18.16b, V19.16b

MOV V20.16b, V21.16b

MOV V22.16b, V23.16b

4. Issue build and setting up command.

python3 setup.py install

That is it. Now you can try loading PyTorch and start experiments in neural network training directly from your phone!

Happy coding anywhere.
How to Combine Textual and Numerical Features for Machine Learning in Python
By: Yuki Takahashi
Tags: Pytorch, Getting Started, AI, NLP, Scikit Learn

1. scikit-learn (e.g. for Tfidf)

When you have a training dataframe which contains both number fields and text and apply a simple model from scikit-lean or some equivalent, one of the easiest way is to use sklearn.pipeline.FeatureUnion in Pipeline .

The following example assumes X_train to be a pandas DataFrame, which consists of many number fields with a text field at the last column. Then you can create a FunctionTransformer to separate the number columns and the text column. The functions passed to this FunctionTransformer can be anything, so modify this according to your input data. Here it simply return the last column as textual feature and the rest as numerical features. Then apply Tfidf Vectorizer on texts and fed into the classifier. This sample uses RandomForest as an estimator and GridSearchCV to search the best model in the given parameters but it can be anything else.

Combine Textual Inputs and Numerical Inputs in scikit-learn (image created by author)

Scikit-learn provides good APIs to manage ML pipeline, which just does the job and can even more complex steps in the same way.

2. Pytorch (e.g. for LSTM, BERT)

If you apply deep neural network, it’s more common to use either Tensorflow/Keras or Pytorch to define the layers. Both have similar APIs and can combine the text and number inputs in the same way and the following sample uses pytorch.

To process texts in a neural network, first it should be embedded in such a way that the model expects. It is also common to have a dropout layer to avoid overfitting. This model add one dense layer (i.e. fully-connected layer) before concatenating with numerical features to balance the number of features. Finally, apply the dense layer to output the desired number of output.

Combine Textual Inputs and Numerical Inputs in a neural network (image created by author)

If you are looking for the end-to-end process how to apply the above LSTM/BERT models in the deep learning, please refer to this post — LSTM vs BERT — a step-by-step guide for tweet sentiment analysis.
Classifying images of different places
By: Luis Salvador Heysen Taborda
Tags: Python, Image Classification, Deep Learning, Pytorch

In this blog post, I’ll describe the process of designing and training a Deep Learning model to classify images of the “Intel Image classification Dataset”, which contains images of six different categories: mountain, forest, buildings, glacier, street and sea. For this project, I used the Python library PyTorch.

Before designing the model, it’s always a good idea to explore the data.

Some random images from the dataset

The images in the dataset have an average size of 150 x 150 pixels, but not all of them have the same size, so I had to resize them. I chose a size of 128 x 128 pixels. After exploring the data, I noticed there were 14,034 images for training, and 3,000 images for testing. Now, it’s time to design and train the model.

My first approach was creating a Convolutional Neural Network. After half an hour of training, my highest accuracy was 70%. I wasn’t happy with this result, so I made some changes that helped improve the training. Instead of using the images as they were in the dataset, I applied some transforms. I did a random crop with a reflection on a 4-pixel padding. I also applied a random horizontal flip, with a probability of 50%. Finally, I converted the images into tensors and normalized them to have a more centered distribution of pixels. After applying those transforms, the images look like this:

Batch of normalized images

I also did some improvements to the training phase. I used what Aakash NS taught in lecture 5 of the course. I used Learning Rate scheduling, which uses a learning rate scheduler to change the learning rate after every batch of training. I also used Weight Decay and Gradient Clipping, which are regularization techniques to prevent the weights and gradients from becoming too large, respectively. After adding these techniques to the fit function, I had very good results.

I trained for 37 epochs, with two maximum learning rates: 0.01 for the first 32 epochs, and 0.004 for the last five epochs. After all this training, which took around half an hour, I reached a final accuracy of 90.39%, a much better result in comparison with the previous accuracy. Here are a couple of plots:

Accuracies:

In this plot, we can see that at the beginning, the accuracy was pretty low, around 1%, but after the first epochs, it raised to 80%, and at some point, it fell to somewhere around 30%. In the last epochs, we can observe that there was a final increase in the accuracy.

Training and Validation Loss:

In this plot, we can see the training and validation loss. Close to epoch 10, there is a sudden jump in the validation loss, which matches with the big fall in the accuracy, in the previous plot. Both the training and validation loss look almost the same, and that means there was no overfitting, that’s good news!
[ML Project] Bone-Age-Assessment(1)
Tags: Side Project, Deep Learning, Pytorch, AI, Machine Learning

Learn more. Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Make Medium yours. Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore
ML14: PyTorch NN—FNN on NMIST
By: Morton Kuo, Yu-Cheng Kuo
Tags: Data Science, Deep Learning, Pytorch, Machine Learning, Neural Networks

ML14: PyTorch — MLP on MNIST

Read time: 20 min Complete code on Colab: https://bit.ly/34yjiod

There are two major areas in DL field, i.e. solving image-related problems (computer vision) and solving text-related problems (natural language processing). Nevertheless, The DL community often casts the spotlight on image-related problems rather than text-related problems since the former is more lucrative.

This article illustrates the workflow of a NN model by the well-known dataset MNIST. We get 98.13% accuracy on test data of MLP on MNIST.
The PyTorch Story: Practice Session 1
By: Himanshu Shandilya
Tags: Linear Regression Python, Deep Learning, Pytorch, Machine Learning, Artificial Intelligence

"Except Love, Everything is Physics".
Exploring FreezeG — style transferring on StyleGAN2?
By: Allen Ng
Tags: Freezeg, Stylegan2, Pytorch, Zeals, Machine Learning

How does it work?

From the repo readme we know that the idea of this work is trying to freeze only the later layers of other pre-trained models, so the higher level layers which handle feature mapping are frozen, but only the style generating layers are enabled for training.

Diving into the code to check out which layers are exactly frozen by bryandlee, by comparing the code with the original repo, we can see this part of the code is added when training generator.

We can see only convs and to_rgbs layers are enabled for training, all the rest is frozen

We can confirm that the generator mapping model is completely frozen during training, so we know that given with a same z, source and target model will generate a same w.

For stylegan2, the right side of the model changed to below structure.

So we can see also only partial layers of synthesis model is enabled for training, Conv layer inside graph (c) and referring to stylegan2’s official source code, to_rgbs layer enabled for training in each block.

Since both source and target model having the same w, with the same feature mappings, in the context of generating an animal image, means the eyes, mouth, ears, etc. position. That’s why the generated images look quite identical but only difference in styles.
The most common PyTorch functions on Colab
By: Mohammed Hamdan
Tags: Basic Function, Python, Pytorch

PyTorch is not a new programming language such as these libraries are a little different from standard libraries like Matplotlib or things of that (feel free to jump to the bellow code- the code is assignment 1 of my Deepleaning Zero to GAN in Jovin).

I mean, they didn’t define because they do define some useful metrics for support graphs and flow. And that is why it’s essential to know these basic primitives, which are supported by them.

You need to care about how they are coded up in Python, actually, in some way, but all you need to care about like senses, and therefore, this PyTorch near what it is.

So, I’m saying that there’s something called a variable; there’s something called constants. And that’s c++ or Java, whatever you have because it is machine learning, you need to have Facebook constants and variables and constants for sure. But in two lines of variables.

You know what kind of variables are things that your data lake Lake, you know, got data and labels.

But other kinds of variables are variables that have weights and biases, so there are learnable variables. Some variables are not supposed to update your data, or you know your neighbor might even update the weights. So that’s the difference between the typical programming language and machine learning programming, and machine learning programming language should support the parameters and constants, whenever variables and, you know, variables like that, the latest one. So, you call them differently in Python, even in terms of going back on TensorFlow because it deals with tensors. The tensor matrix in the high dimensional matrix is called a tensor, and a tensor is a number one dimension. A matrix should consider the tensor, and because of tensors. So, number four is called TensorFlow. We start with some image, for example, which is like a 3D tensor. So, you know, x y and then RGB and, therefore, the third dimension. And you manipulate that image to create a new one and do some operations.

https://gist.github.com/MHHamdan/61547920bf963694ac40e1346cabd814
經典網路系列(六) — Sequence to sequence
By: Mz Bai
Tags: NLP, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
Pytorch實作系列 — FCN
By: Mz Bai
Tags: Computer Vision, Deep Learning, Pytorch

gitE0Z9/classical-network-series

You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or…
Installing Python3.x on Linux from source
By: Hari Manikandan
Tags: Installation, Python3, Python 39, Pytorch

If you ever use deep learning libs, then they probably need to compile some native C++ code. They might call modules from natively compiled C++ libs. So heres a clean way to get py3.x (x can be 7/8/9) installed to use with linux via building from source.

Some variables:

export PYHOME=/opt/py3.x #replace with your path.

export LD_LIBRARY_PATH=$PYHOME/libffi-3.2.1/lib:$PYHOME/libffi-3.2.1/lib64:$PYHOME/lib

Install libffi

It is an interface for calling C compiled libs from python, which can be installed with yum/apt-get. Heres building from source.

wget ftp://sourceware.org:/pub/libffi/libffi-3.2.1.tar.gz

tar xfz libffi-3.2.1.tar.gz

cd libffi-3.2.1

./configure --prefix=$PYHOME/libffi-3.2.1

make

make install

cd ..

Install Python

Substitute x with 7/8/9



tar xzf Python-3.x.0.tgz

cd Python-3.x.0

./configure --prefix=$PYHOME LDFLAGS=-L$PYHOME/libffi-3.2.1/lib64 PKG_CONFIG_PATH=$PYHOME/libffi-3.2.1/lib/pkgconfig --enable-shared

make && make install wget https://www.python.org/ftp/python/3.x.0/Python-3.x.0.tgz tar xzf Python-3.x.0.tgzcd Python-3.x.0./configure --prefix=$PYHOME LDFLAGS=-L$PYHOME/libffi-3.2.1/lib64 PKG_CONFIG_PATH=$PYHOME/libffi-3.2.1/lib/pkgconfig --enable-sharedmake && make install

To make it discoverable, add to path.

echo "PATH=$PYHOME/bin/:$PATH" >> ~/.bashrc

echo "export PYTHONPATH=$PYHOME/bin/" >> ~/.bashrc

echo "export LD_LIBRARY_PATH=$PYHOME/libffi-3.2.1/lib:$PYHOME/libffi-3.2.1/lib64:$PYHOME/lib" >> ~/.bashrc

source ~/.bashrc

Coming up next: Installing pytorch, CUDA drivers, cuDNN for GPU run.

References
ML12: A Detailed PyTorch Tutorial
By: Morton Kuo, Yu-Cheng Kuo
Tags: Data Science, Deep Learning, Pytorch, Machine Learning, Neural Networks

Outline

(1) Setup

(2) PyTorch Basics 01

## 01: A helper function — describe()

## 02: zeros(), ones(), arange()

## 03: rand(), randn(), randint()

## 04: From Python lists to PyTorch tensors

## 05: Tensor types — float, long

## 06: Tensor types — .float(), .long(), .int()

## 07: Tensor types — .type() & .dtype

## 08: PyTorch and NumPy — from_numpy()

## 09: PyTorch and NumPy — .numpy()

## 10: Tensor size — .size() & .shape

## 11: Indexing & slicing — 1

## 12: Indexing & slicing — 2

## 13: Indexing & slicing — 3

(3) PyTorch Basics 02

## 01: Addition — add(), .add_(), .fill(), .zero_()

## 02: Multiplication — 1

## 03: Multiplication — 2

## 04: Multiplication — 3

## 05: More math functions — mean(), sum(), sqrt(), sqrt_(), abs(), acos()

## 06: More math functions — eq(), equal(), ge(), gt()

## 07: Dimension based tensor operations: .view(), reshape(), .t(), transpose()

## 08: Dimension based tensor operations: squeeze() & .unsqueeze()

## 09: Dimension based tensor operations: cat(), stack()

## 10: Dimension based tensor operations: split(), chunk()

## 11: Indexing into storage — 1

## 12: Indexing into storage — 2

## 13: Contiguous tensors — 1

## 14: Contiguous tensors — 2

## 15: Contiguous tensors — 3

## 16: Other functions: .normal(), nonzero(), expand(), expand_as()

## 17: Other functions: argmax(), max(), min(), sort()

## 18: Other functions: eye(), zeros_like(), ones_like()

(4) PyTorch Basics 03

## 01: Tensor on GPU — .cuda() & .cpu()

## 02: Tensor on GPU — to(device= ‘cuda’)

## 03: Tensor on GPU — an error

## 04: Tensor on GPU — to(torch.device(“cpu”))

## 05: Elaborate speed difference between CPU & GPU by matrix multiplication

## 06: Computational graph — 1

## 07: Computational graph — 2

## 08: Computational graph — 3

## 09: Computational graph — 4

## 10: Computational graph — 5

(5) Summary

(6) Reference
5 PyTorch Tensor Operations You Should Know
By: George K. Agyen
Tags: Python, Data Science, Pytorch, Machine Learning, Programming

Function 1. torch.eye

this function creates an n by m dimensional identity matrix. an identity matrix is one that has ones(1) in its principal diagonal and zeros(0) anywhere else. You can only specify the n without the m, it will automatically take m as same as n and create a square matrix with dimension n by n

Example 1:

This creates a 3row x 4column identity matrix as specified in the torch.eye function. we see that the principal diagonal has only ones an anywhere else is zero(0)

Example 2:

In the example above since we specified only the number of rows, it automatically assumed the same number of columns and created a 3x3 identity matrix

Example 3:

If you try to supply a float number for either the row or column parameter it throws an error. This is because it only takes int values

This is an easy and quick way to create a 2d identity matrix tensor. it saves the time of you typing it manually
Pytorch 實作系列 — biLSTM
By: Mz Bai
Tags: NLP, Lstm, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
經典網路系列(13) — prototypical network
By: Mz Bai
Tags: Computer Vision, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
Install conda and set up a Pytorch 1.7, CUDA 11.1 environment on Ubuntu/Windows
By: Felipe A. Moreno
Tags: Cuda, Conda, Pytorch, Nvidia Driver, Nvidia

In this fast post, you will know how to set up an environment using conda (Anaconda) and PyTorch last stable version (1.7.1) with an Nvidia Driver 11.1; first of all, you can check how to successfully install CUDA for ubuntu here, at the first half of that post you can learn how to install any driver-version for your GPU.

My 2 NVIDIA Quadro RTX 6000

Installing Conda

First, you need to install Anaconda on your computer. To do this, you can check the installing guide from the website in any OS here.

Setting up a Conda environment

Then, set up a conda environment ready to use PyTorch, once you installed conda, just type:

conda create --name pytorch python=3.7

conda activate pytorch

conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch

conda install numpy matplotlib scikit-learn pandas tqdm jupyter scikit-image

conda install -c conda-forge tensorboardx

conda install -c conda-forge opencv

conda install -c anaconda scipy=1.5.3

The first line creates our environment called “PyTorch” and you can select the python version (I choose version 3.7).

The second line activates the created environment, in other words, changes our session to our PyTorch environment.

The third line install PyTorch using the Cuda toolkit 11.0, but Are you wondering why I used toolkit 11.0 when my computer has a CUDA version 11.1? — — Well, let me say to you that Anaconda has the amazing option that you can install a Cuda toolkit version less than your driver into your conda environment. Besides you can check versions and Cuda toolkit Pytorch official guide installation.

Then, the next lines install some other libraries that you would use (depending on your task), and always is good to have them installed :)

After all this process, you are free to check your python version inside your conda environment and check if Cuda is available as well.

Checking my torch version and GPU available

So, now you can start creating your model and developing some projects :)

I hope you enjoyed reading this post, if you liked and founded it helpful share it pls.
Some notes on memory in pytorch
By: Rohola Zandie
Tags: Python, Pytorch

You may have been confused by some functions on torch that does the same thing but with different names. For example:

reshape(), view(), permute(), transpose()

Are they really doing differently? No! but in order to understand it first we need to know a little bit how the tensors are implemented in pytorch.

Tensors are abstract or logical constructs just like arrays that can’t be implemented the way they conceived. The obvious reason is that memory cells are sequential, so we need to find a way to save them in memory. For example, if we have a 2d tensor (or array) like below:

The normal (or contiguous) way of saving it into memory is rowed by row. So we will have:

Every tensor has metadata that tells how to read the tensor. For example, in this 2d tensor for accessing the next row we have to move forward 3 steps and the next column we should move 1 step. We call these two numbers strides. So we can extract them like below:

This opens up new possibilities for us because in order to change the matrix we can change the stride metadata! For example, if we change the (3, 1) stride to (1, 3) we actually transposed the matrix without any manipulation in all the memory items.

As you noticed the tensor is not contiguous anymore because we changed it!In order to go to the next row, we only have to skip 1 value, while 3 to move to the next column.

This makes sense if we recall the memory layout of the tensor:

[0, 1, 2, 3, 4, …, 11]

In order to move to the next column (e.g. from 0 to 3, we would have to skip 3 values. The tensor is thus non-contiguous anymore! To make it contiguous just call the contiguous on it:

When you call contiguous() , it actually makes a copy of the tensor so the order of elements would be the same as if a tensor of the same shape created from scratch.

Note that the word “contiguous” is bit misleading because its not that the content of tensor is spread out around disconnected blocks of memory. Here bytes are still allocated in one block of memory but the order of the elements is different!

With the same token, the view function is just a view of the original variable which means if you change the original memory it will change:

This is actually very efficient because we don’t have to make new memory slots for a transformation. But reshape can copy the original data. from the original doc:

Contiguous inputs and inputs with compatible strides can be reshaped without copying, but you should not depend on the copying vs. viewing behavior.

For example if we have code like below:

gives:

The view doesn’t work on non-contiguous data.

Also, consider that permute is another function that only works on the metadata so it also creates non-contiguous data. Permute changes the order of axis so it’s totally different from view or reshape that change the shape of the matrix.
經典網路系列(14) — pix2pix
By: Mz Bai
Tags: Computer Vision, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
經典網路系列(12) — Temporal convolutional network
By: Mz Bai
Tags: Computer Vision, Deep Learning, Pytorch

locuslab/TCN

This repository contains the experiments done in the work An Empirical Evaluation of Generic Convolutional and…
經典網路系列(十) — UNet
By: Mz Bai
Tags: Semantic Segmentation, Unet, Deep Learning, Pytorch

PyTorch for Beginners: Semantic Segmentation using torchvision

This post is part of the series in which we are going to cover the following topics. 1. What is Semantic Segmentation…
經典網路系列(11) — DCGAN
By: Mz Bai
Tags: Gan, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
An unofficial colab walkthrough of Vision Transformer
By: Hiroto Honda
Tags: Colab, Deep Learning, Pytorch, Artificial Intelligence, Computer Vision

An unofficial colab walkthrough of Vision Transformer

Hi I’m Hiroto Honda, a computer vision researcher. [homepage] [twitter]

Playing with codes is often much more effective to understand machine learning methods than reading papers.

This time I have created a colab notebook for the simple walkthrough of the Vision Transformer.

>>>>[colab notebook] <<<<

You can run the cells directly or make a copy of the notebook in your drive.

Schematic of the Vision Transformer inference pipeline from our colab notebook.

We hope you will be able to understand how it works by looking at the actual data flow during inference.

Have fun!

credits
MLventures with PyTorch Functions
By: Hariharan Subramanian
Tags: Deep Learning, Pytorch, AI, Jovian, Machine Learning

Recently I enrolled for an online course on Deep Learning with PyTorch: Zero to GANs from Jovian.ai, in collaboration with Freecodecamp.org.

Being coding-focused, the course does recommend knowing programming basics. No prior machine learning experience is required.

Since September I have been taking the famous Stanford Machine Learning course by Prof. Andrew Ng on Coursera. This certainly helps me with the zero-to-GANs course.

It is indeed a wonderful time to learn ML, given the large selection of online courses and code camps. Everyone has their own preferred style of learning. I can say that not everyone will like the pace and style every course. This one is no exception. But I think with a little bit of patience anyone can get accustomed to this course due to its simplicity behind all of the jargons. I am extremely grateful for Jovian and its team to provide this course free of cost. (No, I am not paid, nor do I receive any benefit from Jovian.ai for writing this!)

I was skeptical if I will be able to follow the course. With 2 lessons and assignments complete, I think I will survive through this. I managed to survive 2020 after all, what’s in another course!

With that intro out of the way… (I promise, future stories will get straight to the point, unlike recipe articles)

The 1st assignment of this course is aimed to get used to PyTorch functions, Google Colab, Jovian Jupyter Notebooks. On that note, here are 5 PyTorch functions from several of them that I tried. And I hope they will get you hooked up to practice PyTorch.
Playing Conway’s Game of Life
By: Meir Dragovsky
Tags: Python, Pytorch, Convolutional Network, Neural Networks, Cellular Automata

CellAuto class

A special case were rules are {live: [2,3], dead: [3]} is called Conway’s Game of Life

There are lot of repetitive patterns that the system can converge into.

from wikipedia

Util class for animating is defined here

Convolution model to predict Conway’s Game of Life next generation

In this section we will train a model to predict next generation.

The data set contains single rule — the Conway’s game of life.

Please note that the 3 convolution layers consists of 3x3 filters gaining information of the direct neighbors.

However unlike typical image processing CNN we don’t need any information more than direct neighbors.

For that reason no down sampling is made except for the last fully connected layer that brings back to input size needed for the loss function.

Training the model

On Each epoch we will generate new data set randomly.

On a 16x16 grid there is very small chance of data repetitions

In fact since the initial random settings of 1’s and 0’s at the beginning of each game is a binomial distribution, calculating its CDF reveals with confidence of 99% there will be at least 11 1’s where the probability is 0.5.

In that case the probability of initial data repetition is at least:

The Loss Function is standard MSE between the input (calculated next generation) and predicted (predicted next generation)

The predicted output is of continues float type while the calculated is of discrete zeros and ones

This loss function is over precise to our needs as we need only minimize L(calculated — round(predicted)) however in our case it ensures faster converge

In the Validation function we generate small data set and calculate the error percentage. In case its drop below 0.5% we finish our training.

After training we can compare our predictions VS. real computed data using the util Animator class:

Training model with multiple rules

Here we are about to upgrade the model to be able to predict cellular automaton next generation with 3 distinct rules.

Each rule applies totally distinct behavior from the model

The model input will be the current generation matrix (16x16) and the rule discriminator vector (size 3)

Please note how the rule vector merge itself into the first convolution layer as separate channel — I found it way much better in terms of accuracy and time convergence than any other configuration such as fully connected between or after the 3 convolution layers.

Reverse Engineering Cellular Automaton

In this section we will train a model to classify the exact rule that produce 2 consequent generations.

This time the input will be 2 16x16 matrices representing the 2 consequent generations and the output vector of size 3 representing the rule class.

Each input matrix assemble to separate channel up sampled and then down sampled

Finally there are 2 fully connected layers to reach its final size

I hope you enjoyed this journey to Cellula Autonom and its CNN Implementation. You can find all the code source here
經典網路系列(九) — Image style transfer
By: Mz Bai
Tags: Style Transfer, Computer Vision, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
5 Useful Pytorch Functions to Get Started
By: Prabhat Bhagel
Tags: Data Science, Zero To Gan, Jovian Ml, Pytorch

This is failed to stack because x and y are not with same dimensions.

Use this function to stack multiple matrixes into a single matrix along its x or y directions.
PyTorch Lightning 1.1 - Model Parallelism Training and More Logging Options
By: Pytorch Lightning Team
Tags: Data Science, Deep Learning, Pytorch, Machine Learning, Pytorch Lightning

Image by Author

Lightning 1.1 is now available with some exciting new features. Since the launch of V1.0.0 stable release, we have hit some incredible milestones- 10K GitHub stars, 350 contributors, and many new members in our slack community! A few highlights include:

Sharded model training- save up to 55% of memory without losing speed

Sequential Model Parallelism

Automatic logging for callbacks and any LightningModule hook*.

Lightning Bolts 0.2.6 release

Sharded model training [BETA]

We're thrilled to introduce the beta version of our new sharded model training plugin, in collaboration with FairScale by Facebook. Sharded Training utilizes Data-Parallel Training under the hood, but optimizer states and gradients are sharded across GPUs. This means the memory overhead per GPU is lower, as each GPU only has to maintain a partition of your optimizer state and gradients. You can use this plugin to reduce memory requirements by up to 60% (!) by simply adding a single flag to your Lightning trainer, with no performance loss.
經典網路系列(八) — Deep Q Network
By: Mz Bai
Tags: Reinforcement Learning, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
PyTorch Useful miscellaneous functions
Tags: Bots, Python, Jupyterlab, Pytorch

This story is about the miscellaneous functions in PyTorch which are useful. I have created other stories on PyTorch basics Part1 and Part2.

In this story we will look at some useful functions which are not covered in part1 and 2.

torch.stack this function is also similar to concatenation but it will create a 3-D tensor from stacking 2 tensors.

torch.mean generates mean value of tensor. A simple but useful function, input to this function is tensor itself.

torch.trace as the function name suggests it calculates trace of the tensor. Trace means sum of its principal diagonal elements. Input to this function is tensor itself.

torch.unique will generate a tensor with unique elements from the main tensor which is passed as input.

Trending Bot Articles:

torch.masked_Select is like a filter applied on a tensor to get only specific values from a tensor based on a position. we have two tensors 1st tensor is a main tensor on which you want a filter and 2nd tensor is the mask tensor where values of it are either true or false.

How does it work? e.g: say i have a 4x4 tensor and i want to extract only values which are greater than 0.3 then we create mask based on 4x4 tensor and apply it using masked_select as in the below picture

torch.where is a conditional statement function. This require three inputs 1st is conditional statement eg: X > 0.4. 2nd is the primary tensor X and 3rd is secondary tensor Y. This will apply the condition on tensor X at every index, if a value does not satisfy the condition then value from Y at same index is sent as output.

We can also perform linear equations in PyTorch. e.g: Y = X*X+3X+2I. where I is identity matrix means only diagonal elements are 1.

Tensor operations in GPU MODE . When you are trying to work on tensors in GPU mode make sure that all the tensors are set to GPU compatible else you will see this error.

Here X was on CUDA(GPU) and Y was on CPU mode so it gave that error. To fix this make y also into GPU mode.

These are some useful functions in PyTorch. This story will be modified in future to add more functions. Thanks for reading till the end.

Story on basics of PyTorch: PyTorch Basics Part1 and PyTorch Basics Part2

Next Story is on Scipy and NLP functions like Named Entity Recognition, lemmatization and stemming, POS tagging and N-grams.

Don’t forget to give us your 👏 !
Neural ODEs with PyTorch Lightning and TorchDyn
By: Michael Poli
Tags: Deep Learning, Pytorch, Differential Equations, Machine Learning, Pytorch Lightning

Hypersolvers in PyTorch Lightning: Faster Neural Differential Equations

Neural Differential Equations inference is typically slower than comparable discrete neural networks, since these continuous models come with the additional overhead of solving a differential equation. Various approaches have been proposed to alleviate these limitations e.g regularizing the vector field such that is easier to solve. In practice, an easier to solve NDE translates into a smaller number of function evaluations — with adaptive-step solvers — or in other words less calls to the neural network parametrizing the vector field.

However, regularizing the vector field may not always be an option, particularly in situations where the differential equation is partially known and specified a prior — as is the case for example in control applications.

Euler method versus its Hypersolver variant — solving a Neural ODE for MNIST image classification. HyperEuler maintains a lower error across the solution, in turn preserving more accurate class predictions.

The framework of hypersolvers considers instead the interplay between model and solver, analyzing ways in which the solver and the training strategies can be adapted to maximize NDE speedups.

In their simplest form, hypersolvers take a base solver formulation and enhance its performance on an NDE with an additional learning component, trained by approximating local errors of the solver. Other techniques are available, for example, adversarial training to exploit base solver weaknesses and aid in its generalization across dynamics.

We will use this HyperEuler implementation to speed up inference of the Neural ODE of the previous section. Here, we will use as hypersolver network g a tiny neural network made up of a single linear layer:

Do not be alarmed by the high number of epochs. These will be simple training iterations, since we are doing full-batch training.

After training, we can visualize the flows of the Neural ODE solved with our hypersolver variant of the Euler method, HyperEuler , and verify whether they are the same as those solved with the adaptive — step method:

Flows of the trained Neural ODE, solved with HyperEuler. The flows are almost identical to the solutions shown above, but are computed in 50% the number of function evaluations.

At the cost of only 12 additional parameters, we are able to successfully reduce the number of function evaluations to less than 20, starting from 40+. Not bad!
經典網路系列(三) — generative adversarial network
By: Mz Bai
Tags: Gan, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…
經典網路系列(七) — ResNet
By: Mz Bai
Tags: Resnet, Deep Learning, Pytorch

gitE0Z9/classical-network-series

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build…