fabiochiu/medium-articles huggingface dataset:
Prisoner 5
By: J. Lee Graves
Tags: Mystery, Serial Fiction, Fiction, Science Fiction, Creative Writing

*Prisoner 5 is a serial science fiction story released every Saturday*

Chapter 3

The numbness returns after a time. Numbness of the body and the mind.

The timelessness of the day to day, hour to hour is torture worse than the beatings. The beatings are real, everything else seems a vague illusion of a nightmare.

The voices in my head, the voices of my neighbors; they blend making it hard to know what’s real. What’s not. Either way I go with it. Questions are questions I have also, so it’s hard to answer wondering if I’m asking myself or if 22 asked. Is there even a 22? Are my speculations based on delusion? My neighbors input, is it also my own thought. Am I really alone and imagining the voices around me? I couldn’t imagine going through this alone. Maybe I don’t want the answer to that question. Either way it’s horrific.

I moved to the corner and sat. Moving still hurt but like the old saying — still alive for what it’s worth. Sitting the numbness set back in. Mind and body.

The grey made it easier to go numb. The floor and the walls came together in a way that blended, seamlessly causing a nauseating illusion at first but at least now my mind has wrapped around the perception. Round, in my head it’s round. A bubble of misery.

The screams, getting further away, echo. The size of this place must be immense. How many tens or hundreds are here? Thousands? Where is here?

Closing my eyes, leaning my head back against the cold wall I inhale deeply. Pressing my back against the wall, the healing cold. Soothing. Now. Now? I wait. Waiting for…I don’t know. Just waiting.

Fire-ants? I can’t stop thinking about fire-ants. Two words I seem to know but I can’t remember them together and why associate with pain? Fire burns. Ants. Burning ants? Ants that burn. Bite. Ants that sting like fire when they bite. Yes! Fire.

It hurts so much to remember. I hold my head, squeezing helps — it hurts. Hurts differently.

I am Prisoner 52781 — 340 but my neighbors call me 5.

Chapter 4
... yes, all good and fine..! But not without Him... :-)
By: Christine Sander
Tags: Science Fiction, Novel Writing, Quantum Physics

... yes, all good and fine..! But not without Him... :-)
2020 Blows
By: Connie Song
Tags: Storymaker, 2020, Ghosts, Fiction, Science Fiction

It’s early Christmas morning and I’m standing on my front porch, soaking in the light rain plopping down on an unseasonably disjointed 54 degree day in December. Tonight is predicted to drop to 24 but for now, a strong gust of north wind passes right through me. It feels raw and chilly. I barely give a second thought to the nicotine rush that used to be part of my morning mantra.

Everything is grey and soggy except for some twinkling colored lights on my neighbor’s veranda. Fat raindrops fall, like pebbles into a pond and the air is clean but aches my joints.

I see four people scurrying into the church down the road. Somehow, my heart leaps with alternate bounds of sadness and joy. I feel frozen in isolation.

Thoughts interrupted, I faintly hear my name and that familiar voice calling to me to come in before Christmas breakfast gets cold. I’m having serious second thoughts about that cigarette, but my pockets are empty. Inside, there’s twisted smiles and wide-eyed wonder, tousled hair and anticipation dressed in red, cozy pajamas. The kitchen is tinged in warm sepia and holds moments of precious memories, if measured only in microseconds.

I see the birds hunting for crumbs. Without an umbrella, I pull my hood tighter to my head and make a dash for the corner store. Until I realize, there’s nothing in my pockets. And the clerk behind the register looks right through me, as if I’m not actually there.
In Service Of —
By: Sean Stubblefield
Tags: Science Fiction, Axanar, Short Story, Star Trek

In Service Of —

Live in service… BE of service… to others… is upheld as epitome of social intercourse.

Why must our lives be commandeered and conscripted in order to attain value… merit?

We can become way too obsessed and upset with “what’s next” mentality. He just did something amazing, and you say “Neat, what’s next?”. Both as creators and audience.

Why can’t we revel more in what has been done? Dwell in it.

We are too quick to rush past.

He could do only one amazing thing, and it be ultimately dismissed as “not enough”, because that’s “ALL” he did? As if he is a has-been. A failure?

Already, he was worn out by war.

But the enlisted midshipman did not join Star Fleet to kill Tlingons.

Because of what they made him―

What they made him DO?

No, what they made him INTO.

Inflicting suffering on another does not undo the suffering they caused. No justice there.

Appointing blame will not absolve, not accrue rectification or reconciliation.

Cosmologically speaking, everyone alive now is already dead.

Yes, but by that logic, they are also not yet manifest.

True― but by either measure, proportionally, they are all non-existent. So what does it really matter in the grand scheme? And why must he be expected to attribute his life to vacancy?

So — during shore leave on Rigel and having had enough-of-that-no-thank you, he decided to abandon his non-commission after a fateful union with a liberated Orion ex-slave woman in a tavern.

HE is alone because he left everyone.

SHE is alone because everyone left her.

Together, they can be alone.

Together, they are more.

Together… they are… NO more.

Her esoteric ramblings made sense, if you bother to listen, to parse, to comprehend. If you know how. And he does. And she does. So they do. A poetic riddle-code for deciphering.

Immersed in a purity ceremony, he and she sat companionably and comfortably in each other’s lap, legs and arms co-crossing and eyes commiserating.

Her placid labia & clitoris grazed his flaccid penis & scrotum; groins touching in a realm balanced between platonic and sexual, separated only by the sheer transparent fabric of their matching leotards.
The most awarded Modern Science Fiction Authors and Novels
By: Hariharan Srinath
Tags: Science Fiction, Awards

I love how Science Fiction literature challenges my conception of what is possible. I started with Isaac Asimov’s Foundation series many years ago and have never looked back, working through the pantheon of all-time greats like Clarke and Asimov and more recently dipping into newer authors via Kindle. Over the holidays, I wanted to expand my exposure to science fiction works in the new millennium and started looking at award winning works.

There turn out to be numerous awards given annually by different science fiction and fantasy societies & conventions in many countries and I decided to focus on some of the biggies — Hugo, Nebula, Locus, British Science Fiction Association, Arthur C Clarke and the John W Campbell awards. Long story short, I aggregated the data in these Wikipedia pages into a normalized dataset to search for my next novel(s) to read.

Most Awarded Novels in the new Millennium

Ann Leckie’s debut novel Ancillary Justice which literally re-invented the space opera genre with it’s complex past/present and individual/group plotlines leads the pack. I am also happy to note that another one of my all-time favorite novels, Rainbows End by Vernor Vinge makes the list. Every passing year, I feel our world growing closer and closer to the one depicted in Rainbows End.

I have read Paolo Bagciagalupi fantastic biopunk novel, the Wind-Up Girl which set in an exotic future Thailand in a world where BioTech “calorie companies” dominate world politics. Neil Gaiman the fantasy genre author most famous for The Sandman series of graphic novels and American Gods rounds out the list of authors I’m familiar with.

Most Nominated Novels in the new Millennium

Another interesting way to cut the data is to look at novels that were nominated for awards but did not win the top awards. Among these, my favorite is the Ninefox Gambit and Revenant Gun, a genre-bending military science fiction/space opera series by Yoon Ha Lee, a mathematician turned novelist. Very challenging initially, these reward the reader with really surprising concepts.

Another really interesting novel in this set is Kim Stanley Robinson’s 2312 which deals with economics & politics in a solar system widely from the Vulcanoid asteroids near the sun to the moons of Saturn and people bio-hack their gender identity. It outlines the ideas involved in colonizing other solar-system planets and asteroids particularly well.

Next Steps

I have just started reading The City & the City by China Miéville and also plan to start on Connie Willis’s works.

Dataset

Here’s the dataset. It’s actually not limited to just the last 20 but goes all the way back to how long information is available in Wikipedia and covers the more “classic” era as well. One point to note about the data is that for novels that are collaborations by more than one author (most commonly Larry Niven & Jerry Pournelle), I have retained only the first author’s name since my intention was to find new novels to read. I may revisit this decision in the future.
Goldie 1
Tags: Science Fiction, Family, Novel

Goldie was a big star. “Goldie”. Her name was actually Demi Gibbon- Ramos. When she was Demi Gibbon all she wanted was to see how people connected. To Demi, life was a complex puzzle, and everyone fits in their exact spot BECAUSE of their relationships with other people. She always wanted to know what shape her puzzle piece was and where it would fit. Does it fit here?

“Goldie” is a house-hold name. She’s not as new or as young as the other players in her league, but she has staying power. She’s been “Goldie” since she was 12 when she “got her start” on a talent search show for kids. No one tells you that you start “getting your start” at least 5 years earlier than that, and that’s if you’re lucky. But Goldie has something. She understands those ridges and grooves that make people stick together, and she understands that some people try to force themselves to fit into a space that isn’t made for them. That throws off the entire puzzle.

Goldie writes and sings about what she sees in those spaces between, and people love her for it. There’s something intimate and holy about the way she writes, and that’s timeless. At least, she hopes it is. She writes about love, and friendship, and family, and the antithesis to all those things. It’s not about what you do for a living, it’s about what you ache for — and if you’d ever have the courage to chase that dream. It’s not about how old you are — it’s about if you’re willing to look like a fool for love, for the adventure of being alive. Everyone understands this at a basic level. Everyone feels this, it just takes someone special, like Goldie, to find the words.

Music flows through Goldie as easily as a hot knife melts butter. She sees inspiration around her every day without even trying. All she ever has to do is connect. Connecting with another person is what life is all about, and if she has the opportunity to do that, to REALLY do that, even for a moment, she takes it every time. She never gets bored with the complexity of a human being.

However, she is a celebrity now. There are millions of people who would love to make that connection with Goldie. If only she could, maybe she could see the big picture, the big, broken, puzzle. This is her biggest woe. She can never reach out enough. She can never connect to everyone, and because everyone wants to connect with her, she can connect with no one, not really. How do you get a moment alone with someone? And if you do, they think they know you, and their expectations sully the real feelings that lie there.

Sure, she’s got staying power, but her supply is running out.
Goodies For Fans of Post-apocalyptic and Dystopian Sci-Fi
By: Rebecka Jäger
Tags: Books, Giveaway, Dystopia, Science Fiction, Thriller Suspense

5.0 out of 5 stars Best Dystopian thriller ever!

“What can I say about Unholy Warrior? There are so many praising cliches out there that certainly fit, but do they mean anything anymore? Let’s skip them and jump right into the story.

Rebane Nordstrom, the MC makes for an unlikely protagonist. Although she has many admirable qualities — she’s a fighter, she’s courageous, she is determined to let nothing get in her way — she is also a cold-blooded killer. But it’s war, and if she wants to live, she must be able to do anything necessary to survive. A highly skilled sniper for the Spetznaz–the elite of the GRU, Russia’s Military Intelligence Agency, she is a valued prize for the enemy, the European Union.

When Rebane is captured, she knows she faces torture, unspeakable horrors, and in the end — death. She is taken to a formidable prison where she meets Major Weisser, a giant of a man capable and willing, to do anything to get answers. And to satisfy his twisted needs. The author, Rebecka Jäger takes us inside the mind of Rebane, and I for one was with her every step she took through hell, in her goal to reach the sanctity of her father’s cabin. Miss Jäger is an artist and paints a vivid picture of a devasted world, one that even our worst nightmares pale in comparison. And in Rebane, one finds a realistic character who must fight everything from the harsh arctic weather to a man who has an army behind him and won’t let go. Kudos Miss Jäger!”

BUY BOOK (AMAZON)

BUY (BARNES AND NOBLE)

BUY (GOOGLE PLAY BOOKS)

Subscribe to today’s bestselling Dystopian authors for a chance to win a paperback of The Stand by Stephen King, Winter World by A.G. Riddle, After the Flood by Kassandra Montag, The Starless Sea by Erin Morgenstern, plus, a Handmaid’s tale enamel pin, a banned books notebook, a set of Fallout coasters, a double-sided Shatter Me bookmark, a Hunger Games travel mug, and a dystopian dreams candle. (For US-dwellers).

ENTER THE GIVEAWAY
Lord Asriel was Supposed to Appear in the Second Season of His Dark Materials
By: Jash Kolze
Tags: His Dark Materials, HBO, Young Adult Fiction, Science Fiction, Fantasy

The HBO series His Dark Materials is an adaptation of a trilogy of the same name by Philip Pullman. I was extremely excited when I first heard about HBO attempting to adapt the trilogy as it deserved a good adaptation. I was also excited that Dafne Keen (Lyra “Silvertongue” Belacqua) was getting more work.

I have read the entire trilogy a couple of times. It seems that every few years I am drawn back to reading the books. Suffice it to say that there have been more than a few things I’ve been excited about seeing on the small screen.

One of those things is what Lord Asriel’s big plan is. Especially where he went after The Golden Compass (or Season 1 of the show). In the first book I thought of him as a villain but by the end of the trilogy if he’s good or evil is harder to determine. He, basically, fights for everyone to remain free so I can’t hate his cause.

Where Lord Asriel went after Season 1 should be amazing if the show continues to do a good job at adapting the source material. Which, after seeing Mrs. Coulter controlling the Spectres, is something I have absolute faith in.

So I got excited for a moment when I heard that Lord Asriel was supposed to have his own standalone episode in His Dark Materials Season 2. Then that excitement died when I found out the episode was cancelled due to Covid-19.

But was it better that viewers didn’t get to see Lord Asriel in Season 2?

His Dark Materials Season 2 was largely based on The Subtle Knife, the second book in the trilogy. A book that Lord Asriel didn’t appear in. The Subtle Knife worked just fine without Asriel’s appearance as it centered around finding the titular weapon.

Changing things up in an adaptation isn’t a bad thing as long as said changes keep to the heart of the source material. Such as Will Parry didn’t appear until The Subtle Knife but had a decent sized plot in His Dark Materials Season 1.

If the plan was for Lord Asriel to appear heavily in the second season of His Dark Materials, I would be against that as his appearance would be too much of a distraction. However, Asriel’s appearance was to be limited to one episode. Thus he wouldn’t be a big distraction and I wouldn’t mind him getting screen time in the second season.

All in all Lord Asriel not appearing in His Dark Materials Season 2 doesn’t matter much.
Film Review: Wanuri Kahiu’s “Pumzi"
By: Eric Rugara
Tags: SciFi, Movie Review, Film, Science Fiction, Movies

Film Review: Wanuri Kahiu’s “Pumzi"

I just watched Pumzi, a sci-fi that makes you think about the environment and what we are doing to preserve it before it is too late.

It has been predicted that one day water will be such a scarce resource wars will be waged over it. Pumzi takes place 35 years after World War 3, known as “the Water War". There is no tree left on Earth. All that remains is desert.

The action takes place in a futuristic structure where lives survivors. Life there is full of order and control. Everything is closely monitored. Your urine is purified into drinking water.

The main character is a woman who has a bad habit of dreaming about a tree. This is treated as an illness and she has to take dream-suppressing drugs.

She works at a lab and on this day she has been sent a soil sample from an unknown source. On examining it, she realizes it has water content and it is not radioactive. Her superiors order her to get rid of it, but she is adamant.

She illegally leaves the facility with a small plant she has started to grow and heads out in search of the soil sample’s source.

We hope this is how her dream comes true. That she will plant the seedling and it will become a majestic tree.

It’s a story about hope and those who have it, those who water it tirelessly, those who do not give up, those who do not listen to the voices that would drown it out, those who believe in the beauty of their dreams.

The abrupt ending informs me that this is a short story adapted into a movie. Wanuri Kahiu could have written and published this as a short story and it would have won prizes; she made a movie instead and it won awards.

The story is excellent, the visuals magical — as magical as a cold, soulless sci-fi world can be. There is a look to this world and its objects that gives you a sense of animation, like how Osborne Macharia’s photography has a hyperreal effect. Arresting beauty. And I suspect the dream shot of her reaching out to the tree inspired Black Panther’s dream scene of T’challa with his dead father under a tree. If not, it did remind me of it anyway. Both films are in the afrofuturism genre.
What are the differences between MindSpore, TensorFlow, and Pytorch?
By: Anıl Arı
Tags: TensorFlow, Ml Platform, Mindspore, Machine Learning Tools, Pytorch

Most data scientists are at least familiar with how R and Python programming languages are used for machine learning, but the possibilities do not end there. Machine learning and AI tools are often software libraries, toolkits, or suites that aid in executing tasks. As with machine learning algorithms, there is not necessarily a “best” AI or machine learning tool. What you use will (and should) depend on the task you are trying to perform.

MindSpor, Tensorflow, Pytorch are three frameworks that are providing machine learning capabilities to applications. They are providing load and process data, training- reuse, and deploying models to devices and operating systems

MindSpore is supported by Huawei, TensorFlow is supported by Google, Pytorch is supported by Facebook. All of them are open-source and you can contribute and develop functionality for these frameworks.

Features and supported platforms are changing due to frameworks and using purposes.

MindSpore Platforms

MindSpore supported languages are Python, C++, and also Java support for the lite(mobile) model. It supports ASCEND, CUDA, and standard CPU platforms. It also gives us commands which we can use a variety of selected options.

Tensorflow Platforms

Tensorflow supported languages are Python, C++, Java for the lite model, Swift for IOS, Go, and Haskell. Their charts are showing us which platform we can load and process data, build train reuse models, and deploying process

Pytorch Platforms

Pytorch supported languages Python, C++. It is similar to Mindspore, It shows only CUDA platform, language, and commands to specified system commands.

Conclusion

There are plenty of open-source frameworks that are loading processing data, building training and reusing models, and deploying them for daily use cases. However, these are for those who want to join, use, and contribute frameworks and use them with their specific platform purposes.

Framework choice can be changed which platform and what operating system developer should use and what model should be supported to their users.

Here you can find a useful article for using MindSpore Custom model on your application:

Thank you!

References
Install TensorFlow 2 with GPU Support (Dec 2020) for Windows 10 using Conda
By: Millan Sanchez
Tags: TensorFlow, Gpu, Nvidia, Windows 10, Machine Learning

Install NVIDIA Drivers for your GPU.

Install MiniConda for Python 3.8 as TensorFlow at this moment doesn’t support Python 3.9:

Once installed with default options, open the Anaconda Prompt from the applications menu in Windows 10.

First lets install Jupyter Notebooks:

conda install -y jupyter

Then we will create a Virtual Environment for Python 3.7 and TensorFlow:

conda create --name tensorflow-gpu python=3.7

(Python 3.8 is supported by TensorFlow at this moment, but as you will see it will not work to do an automated install)

and then activate the environment by using the following command:

activate tensorflow-gpu

Now lets install Jupyter Notebooks support to this environment:

conda install nb_conda

and agree to install the required components.

Theorically by running the following command the TensorFlow GPU package gets installed with all CUDA and CUDNN libraries but it is not working as Dec 2020 (don’t run this command):

conda install tensorflow-gpu #do not run this command

Currently conda install tensorflow-gpu installs tensorflow v2.3.0 and does NOT install the conda cudnn or cudatoolkit packages. Installing them manually (e.g. with conda install cudatoolkit=11.0) does not seem to fix the problem either.

A solution is to install an earlier version of TensorFlow, which does install cudnn and cudatoolkit, then upgrade with pip. If you follow these steps you don't need to install CUDA or CUDNN in your system using the installers from NVIDIA.

conda install tensorflow-gpu=2.1

pip install tensorflow-gpu==2.3.1

With the conda install tensorflow-gpu=2.1 command it is required to have Python 3.7 and that is the reason we didn’t install Python 3.8 when creating the environment.

Now we want to be able to select this new environment as a Kernel from Jupyter Notebooks. For this to happen run the following line in the prompt:

python -m ipykernel install -- user -- name tensorflow -- display-name "Python 3.7 (with TensorFlow GPU)"

Now lets install some basic and popular libraries to test our environment:

conda install pandas

conda install scikit-learn

That’s it. We are finished.

To test run the following code in a Jupyter notebook, and be sure to select the newly created Kernel:

import sys import tensorflow.keras

import pandas as pd

import sklearn as sk

import tensorflow as tf print(f"Tensor Flow Version: {tf.__version__}")

print(f"Keras Version: {tensorflow.keras.__version__}")

print()

print(f"Python {sys.version}")

print(f"Pandas {pd.__version__}")

print(f"Scikit-Learn {sk.__version__}")

gpu = len(tf.config.list_physical_devices('GPU'))>0

print("GPU is", "available" if gpu else "NOT AVAILABLE")

If all goes well you should see the following output:

Tensor Flow Version: 2.3.1

Keras Version: 2.4.0



Python 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)]

Pandas 1.1.5

Scikit-Learn 0.23.2

GPU is available

as you can see GPU is detected and available.

Thanks for reading.
Visualize Machine Learning metrics with Tensorflow and Tensorboard
By: Armin Hadzalic
Tags: Data Science, TensorFlow, Data Visualization, Artificial Intelligence, Machine Learning

Graphs for accuracy and loss in Tensorboard

Let’s come to a workflow that has prepared data in a previous step.

The data may have been extracted on the basis of a CSV file. It does not matter for further consideration, thus the PreProcessing part can be skipped here. It is only important that the data already contains the features Xn and the corresponding label Yn.

Main program could look like this:

# Main program

x, y = PreProcessing(rawfile)

ExecuteNeuralNetwork(x, y)

Let’s dive into the details of the NeuralNetwork method:

code example for Tensorboard logdir by author

Here we see some important processes and hyperparameters that are essential for training. First, the data is separated into a training and test portion (line 8). This is done with a function from the SKLEARN library and the corresponding test_size parameter. In order to get a shuffle, the random_state-parameter is set to, e.g. 42. Taking zero instead, the list would be used as it is, i.e. the first 80% would be processed as training data, the remaining 20% as test data. This can sometimes have an unfavourable effect on the result.

In our example, a neural network is used as the underlaying architecture and algorithm (line 15–20). Of course, other methods such as RandomForest or the DecissionTree-Classifier are also conceivable. Here, one must proceed according to the objectives and prepare the choice very carefully. However, it is not the subject of this post, but you can find the examples for mentioned algorithms on my GitHub path

The only important thing for our consideration is that the results of the training end up in our Tensorboard object and its log directory defined in the callbacks-parameter (line 29, 32). It can be helpful here that all runs are time-stamped and concatenated with other hyperparameter strings. This makes it possible to see at a glance which parameters were used in the model and to display them more easily in the dashboard. In addition, the creation of subfolders in the log directory can conveniently also be used for the historical display of data in the Tensorboard dashboard.

additional metrics if validation_data is used

In order to also get the validation data metrics, one must set the parameter in line 32 accordingly. They will appear as separate fields in the Tensorboard dashboard.
Medical Image Dataloaders in TensorFlow 2.x
By: Prerak Mody
Tags: TensorFlow, Deep Learning, Image Segmentation, Dataloader, Medical Image Analysis

Step 2 — Dataset Transformations

Step 2.1 —Map: Extract subvolume

The cached CT volumes from the generator function are turned into smaller volumes using border information within the meta1 tensor. The mask data is also one-hot encoded using tensor operations for efficiency. This is essentially a map operation performed using the tf.data.Dataset.map() function.

The input to the map function is another function that is decorated with the tf.function decorator. This allows your custom-defined functions to be included into the tensorflow graph thus leading to performance improvement (more on this in the second post). Another input to the map function is the num_parallel_calls that can be used to leverage under-the-hood parallelization optimizations. One can set this value to a fixed number of threads or simply use tf.data.AUTOTUNE to dynamically let Tensorflow figure out how many CPU threads are up for grabs.

Note: An important consideration while using python functions with the tensorflow decorator is to pass tensor objects and not python objects as much as possible. Passing python objects (which may change value during each function invocation) leads to Tensorflow seeing a different function signature and hence converting each invocation to a new node in the tensorflow graph, something that is pointlessly repetitive in nature. This is referred to as the polymorphic nature of the tf.function decorator. Tensorflow graphs and the tf.function decorator shall be explained in detail in the second post.

Step 2.2 — Filter: Background subvolumes

Since CT/MR volumes of the HaN region contain a lot of background-only areas, we would only like to sample a fixed percentage of them (for negative sample training). Using the tf.data.Dataset.filter() function we pass another function that returns True or False .

Step 2.3 — Preprocessing and Data Augmentation

CT volumes have a unit of HU (Hounsfield Units) which measures the radiodensity of voxels in 3D space. The have a range from -1000 (Air) up to 2000 (Bone). For numerical stability, this data should be normalized.

For data augmentation, the repository has an example for 3D rotation, but other operations such as 3D deformation and scaling can be also be used.

Pseudo-code for map and filter operations is shown below.
Don’t be fooled by the loss that I got
By: Liam Pearson
Tags: Data Science, TensorFlow, Loss Function, Keras, Neural Networks

Don’t be fooled by the loss that I got

You’ve just trained a model. The training vs validation loss graph appears on your screen and FINALLY those 2 lines are sitting on top of eachother i.e. there is no overfitting. But be careful….that training loss you see isn’t quite right…

Photo by Dan Freeman on Unsplash

TL;DR: the training loss appears larger than it actually is since it is just an average.

Introduction to the Problem

For context: my model is using a binary target to predict the probability of an event occuring. The loss we are trying to minimise is binary cross entropy.

Take a look at the output below from a Keras/TensorFlow training job. This plot is a common check of how the model’s loss reduced overtime. We want the loss to reduce as well as the training vs validation lines to eventually meet (otherwise it indicates overfitting or underfitting). This graph looks pretty good and is a good first-check of our model.

But when we manually calculate the loss I get a different train loss:

Manually calulating the binary_cross_entropy. Code is at bottom of article.

This is where the confusion begins!! If we compare the TF/Keras loss numbers to ours, we see the validation numbers (in yellow) match. But why do the training numbers (in pink) differ?

This is a cause for concern; while our graph and model-training output indicate there is no overfitting, our numbers suggest there IS overfitting since the training loss is smaller than the validation. Is Keras broken? Is TensorFlow broken?? Can it be trusted? What is happening??

An Explanation of the Keras Training Loss

The loss during training is an average of the loss at each batch.

One of the hyperparameters in your model is batch_size. This is the number of rows that are fed into the model at a time. If you have 500k rows, and a batch size of 500, then for each epoch, the model will train with 1000x batches. At each batch, the model’s loss is calculated (i.e. we will have 1000x losses) and at the end of each epoch, the average is displayed in the training logs.

So why is the training loss higher?

As each batch passes through the model, the model’s weights are updated. Hence, if we have 1000x batches, the model’s weights will be updated/improved 1000x times. The first few batches will tend to have a higher loss than the last ones since the model is continually learning. Hence the average will be slightly inflated due to those higher losses earlier on.

Okay, so what if we just send ONE batch through?

This would be achieved by setting our batch_size = length of data. But the training loss is still slightly different. This is due to a number of regularisation hyperparamters that are switched ‘on’ during training an switched ‘off’ during scoring. So even if we feed in all the data in one hit, the loss displayed during training will still be slightly off. This is documented on the keras website HERE.
Review: TensorFlow Certificate Program
By: Imran Kocabiyik
Tags: TensorFlow

I just received the TensorFlow Developer Certificate.

It is a new program ( announced in March 2020), and I think I am one of the early-holders of the certificate. Here are my impressions and expectations:

Why Choosing TensorFlow Framework?

To have hands-on experience in a deep learning framework, and I have chosen TensorFlow. It is for two reasons:

In general, it is the most popular DL framework.

It is mostly used in the industry than in academia. So, it fits my profile.

Is the Certificate Worth Paying?

The exam costs $100 and takes five hours to complete.

I haven’t proven it yet, but I would expect that it would be useful for demonstrating coding skills.

The Exam Preparation

The certificate program suggests TensorFlow in Practice Specialization in Coursera. The specialization contains four courses:

Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning

Convolutional Neural Networks in TensorFlow

Natural Language Processing in TensorFlow

Sequences, Time Series and Prediction

The specialization program assumes that you have a good understanding of deep learning because this specialization is not about theory but application.

Deep Learning with Python by François Chollet can also be a useful resource.

Technical Requirements for Taking the Exam

You can take the exam with a PC. They consider training the models can take time, but 5 hours should be sufficient.

I purchased the exam with my MacBook because I need to upload a copy of my passport and take a photo with the webcam. However, I started the certification exam on my workstation with a GPU. In summary, GPU is not crucial, but speed doesn’t hurt. 🐢 🐇

You need PyCharm for the exam. The exam plugin creates an environment for you.

Some Suggestions

For preparing for the exam:

Better to start with a blank notebook and start coding from scratch, instead of just running the cells in the given in a tutorial.

Finding new challenges you may like. For example, if you have a binary classification problem in a tutorial, you may consider finding another problem in the real world and work on it.

Explore documentation, change the network architecture, observe how it is changing the results. If you see a new term for the first time (for example, the Huber Loss function was new for me), do some quick research and check how it is working.

During the exam:

To use time efficiently, you may consider creating checkpoints (saving weights in an h5 file and initializing the weights before a new training. Note that this may not work if you change the network architecture.)

You can also use other tools if you feel more comfortable. For example, I developed the model in Jupyter Lab, and then I moved the project into PyCharm.

I hope this helps anyone who might consider taking it.
Installing TensorFlow 2.4 on MacOS 11.0 without CUDA for both Intel and M1 based Macs
By: Chirag Daryani
Tags: MacBook, Apple Silicon, TensorFlow, Deep Learning, Machine Learning

The two popular deep-learning frameworks, TensorFlow and PyTorch, support NVIDIA’s GPUs for acceleration via the CUDA toolkit. This poses a problem for deep-learning development on Macs.

But not anymore!

Along with the latest release of its custom made Silicon Chips based on the ARM Architecture, Apple announced a new ML Compute framework which can be used for accelerated training of ML and DL Models taking advantage of the GPUs present on the Apple Macs without the need of CUDA support.

We now have a Mac-optimized version of Tensorflow 2.4 which we can be installed on both the older Intel-powered Macs and the recently launched M1-chip based MacBooks.

There have been claims that, Tests of the optimized TensorFlow library on several popular neural network benchmarks show “dramatically faster” training times compared to the standard code, with an up to 7x improvement for the optimized library on Apple’s new M1 hardware.

Let’s look at how we can install and test this newest Tensorflow library on our Mac Machines.

Note that before beginning your installation, ensure that you have updated to the latest macOS 11.0 (Big Sur) and have Python 3.8 (available from the Xcode Command Line Tools) installed in your system.

Open the terminal and follow the steps as shown below.

Step 1. Create a Virtual Environment with Python 3.8

From the terminal itself, go the the directory you want to install this environment in and then type the following command.

Note that I am installing it in the Home Directory (/Users/chirag/) itself.

python3 -m venv my_env

This will create a new virtual environment “my_env” with Python 3.8. We will need to specify the path of this environment while doing the Tensorflow installation in the step below.

To check if the required version, that is Python 3.8 was installed, you can activate this environment and then confirm by typing the following command. source my_env/bin/activate

python --version

Step 2. Run the Installation Script (taken from the apple/tensorflow_macos repository )

While installing, it will ask you for a path to an existing Python environment. Specify the path of the environment you created in Step 1.

For my system, the path is as follows.

/Users/chirag/my_env

This will install Tensorflow2.4 in our system in the virtual environment “my_env”

Step 3: Activate the environment.

source my_env/bin/activate

Step 4: Check if TensorFlow installation was done successfully.

Enter your python shell and import Tensorflow by typing the following command.

python3 -c 'import tensorflow as tf; print(tf.__version__)'

As you can see, without any error, we got the output. It means we have successfully installed the MacOS Optimized Version of Tensorflow!
Running content of Jupyter notebook cell only once.
By: Adrian Szatmari
Tags: TensorFlow, Jupyter Notebook, Loading Data, Python, Colab

Sometimes is just happens that you want to run the content of a cell only once.

For instance, it might be that you have some heavy computation in a cell, and you don’t want to rerun it by mistake. Or it might be that you don’t want to reload the data again and again.

Just do it like this:

The first time the cell is run, DATA_IS_LOADED does not exist at all yet, so the “except” section is executed. Then DATA_IS_LOADED is created as a variable, so all the subsequent times, the “try” section will be executed, and the data will not be reloaded.

Hope it helps someone.
How to Train a Neural Network Classifier on ImageNet using TensorFlow 2
By: Pedro Sandoval-Segura
Tags: Classification, TensorFlow, Imagenet, Image Classification, Neural Networks

Overview

There are essentially 3 steps which we’ll work through: preparing the ImageNet dataset, compiling a pretrained model, and finally, evaluating the accuracy of the model.

First, let’s import some packages:

Now, we’ll download ImageNet labels and specify where our ImageNet archive files are located. In particular, data_dir should be the path where ILSVRC2012_img_train.tar and ILSVRC2012_img_val.tar are located. And write_dir should be the directory where we’d like to write extracted image content. Make sure your write_dir directory contains extracted , dowloaded , and data directories.

The key here is that we called tfds.load with keyword arguments to the download_and_prepare call, specifying where our archive files were located and where extracted records should be placed.

Now, because pretrained classification models take 224 x 224 images as input, we’ll need to do some preprocessing of our data. Here we’ll use mobilenet_v2.preprocess_input(i) , but if you’re using a different model, you can replace this call. For example, if I were using a VGG-16, I’d instead call vgg16.preprocess_input(i) .

Next, we compile a model of our choice. In my case, a MobileNet V2.

Finally, because of the way we’ve set up our dataset, we can evaluate this model on the training data and print the accuracy using a few lines of code!
Step By Step Installing TensorFlow 2 on Windows 10 ( GPU Support, CUDA , cuDNN, NVIDIA, Anaconda)
By: Priyanka Singh
Tags: TensorFlow, Deep Learning, Gpu, Nvidia, Windows 10

It's easy if you fix your versions compatibility

System:

Windows-10

NVIDIA Quadro P1000

Step1:

Download & Install ( I used 2019 Version Latest at the time of writing)

https://visualstudio.microsoft.com/vs/community/

Step2:

Get the NVIDIA right driver and install

(For me it is NVIDIA QUADRO P1000)

https://www.nvidia.com/download/index.aspx?lang=en-us

Step3:

Download and Install Cuda toolkit

I downloaded 11.1 version

https://developer.nvidia.com/cuda-toolkit-archive

Step5:

Download and Install cuDNN

( Version compatibility must, I used 8.0.4 for 11.1 Cuda toolkit)

https://developer.nvidia.com/cudnn

Step6:

unzip cuDNN folder

go to cuda/bin directory

copy all files

go to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin

pate copied files here

go to cuda/include directory

copy all files and paste it inside

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include directory

go to cuda/lib and copy all files

paste copied files inside

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\lib

Step7: Important alert

Go to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin

rename cusolver64_11.dll to cusolver64_10.dll

Tensorflow will look for cusolver64_10.dll and can not find it if you do not rename it.

Step8:

Open windows command prompt

Set environment variable path

SET PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin;%PATH%

SET PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\extras\CUPTI\lib64;%PATH%

SET PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include;%PATH%

Step9:

Download and Install anaconda

https://www.anaconda.com/products/individual

Step10:

open anaconda command propmt

create a virtual environment and activate

conda create — name tfgpu python=3.8.0

conda activate tfgpu

pip install tensorflow-gpu

This command installed tensorflow_gpu-2.4.0 version for me

Step11:

Check if tensorflow installed with gpu

import tensorflow as tf

Command below will print which gpu it is using

tf.test.gpu_device_name()

Check https://www.tensorflow.org/install/gpu if version compatibility issue
Gan’s, the next big thing?
By: Parag Chaudhari
Tags: AI, TensorFlow, Gans, Generative Adversarial, Pytorch

Gan’s are coming up as the next interesting field here in ai. Networks that don't consume data but generate it.

Wikipedia defines gans as “A generative adversarial network (GAN) is a class of machine learning frameworks designed by Ian Goodfellow and his colleagues in 2014. Two neural networks contesting with each other in a game (in the form of a zero-sum game, where one agent’s gain is another agent’s loss)”.

So what is the big fuss about gans and why should we study them?

A question I've been asking myself, again and again, they are difficult to train and takes ages to do so, and you better own a chunky graphics card if you dream of creating anything remotely useful. Modern solutions like google colab relieve this problem a bit but the primary question remains, why Gans?

Well for starters they create data.

Digest that for a while, a machine that can dream of completely original never before seen pictures, or never before heard music.

Check the following links if you don't believe me.

Two examples where a previously impossible concept has been realized. What can one do with such a generative network?

Researchers have found this kind of network useful in a wide array of applications, from ai based upscaling, creating completely new pictures, texture translation, or creating your own music.

Let's understand gans first, as the name suggests Generative adversarial network, has two networks competing against each other each becoming smarter every epoch.

The two networks consist of the generator who generates increasingly good counterfeits, and the discriminator whose job is to tell if the image shown to it is real or fake. Just like a currency forger, and a government employee tasked with detecting fake notes. Initially, the notes created by the forger are visibly fake but in the due course of time, it learns and gets better at the job, so good that real humans aren't able to distinguish between them.

All this works on the belief that there exists a probability distribution describing the kind of data we try to generate. For example, if we can generate a vector from the picture of a cat then it is safe to assume the values around that vector may also generate cats.

The one described above is a very simple implementation of gan, of course, more complex implementations like dcgans, wgans, cgans have come out each with its own nuances and features, to be discussed at another time.

One of the most exciting implementations that have caught the eye of every one is deep fakes, where the machine learns to augment any face onto a subject with scary accuracy.

This seems like a nice spot to stop and think about what you have read until now, hopefully, I do keep writing more of these articles on gans as I've taken a liking to them.

Have a good day……..
Tensorflow NLP — Doğal Dil İşleme ve Kelime Tahmini
By: Muhammed İki̇nci̇
Tags: Yapay Zeka, TensorFlow, Neural Networks, Artificial Intelligence, Derin Ogrenme

Learn more. Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Make Medium yours. Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore
Some performance drawbacks of Tensorflow Datasets
By: Maciej Skorski
Tags: Data Preprocessing, Performance, TensorFlow

Tensorflow, the popular framework for machine-learning, recommends its new dataset API for preprocessing and serving the data. It offers many useful tricks, such as caching data in memory, prefetching in parallel threads and others described in this tutorial. However, Tensorflow has some issues with slow data slicing, so that the dataset API may actually do harm in setups where computations are relatively fast; this includes linear models, word2vec and other shallow neural networks. In this note I illustrate this issue on training a classifier on MNIST dataset (handwritten images). As shown in the benchmark below, sometimes it makes sense to serve your data by custom generators not the recommended api.

## model: Logistic Regression



w = tf.Variable(tf.random.normal(shape=(28*28,10),stddev=0.1),trainable=True)

optimizer = tf.optimizers.SGD(0.01)



@tf.function

def train_step(x, y):

with tf.GradientTape() as tape:

all_logits = tf.matmul(x,w) # (n_batch,n_class)

y_logits = tf.gather(all_logits,y,batch_dims=1) # (n_batch,)

logp = y_logits - tf.reduce_logsumexp(all_logits,axis=1)

loss = -logp

gradients = tape.gradient(loss,[w])

optimizer.apply_gradients(zip(gradients,[w]))

Now we compare two ways of serving the data: by custom generators and by Dataset API (with caching and prefetching for better performance).

## serving data: a) via custom generator b) via TF Dataset API



def gen_batch(X,n_window):



def gen():

for i in range(0,len(X),n_window):

yield X[i:i+n_window]



return gen



def gen_data():

n_window = 32

return zip(gen_batch(x_train,n_window)(),gen_batch(y_train,n_window)())



tf_data = tf.data.Dataset.from_tensor_slices((x_train,y_train))

tf_data = tf_data.batch(32,drop_remainder=True)

tf_data = tf_data.cache()

tf_data = tf_data.prefetch(1)

Below running the comparison, the graph and dataset are warmed by one full-pass (then the graph gets built and the api pipeline is cached). Both approaches fit the same classifier, but the code with custom generator runs 50% faster. For the full code, see the jupyter notebook.