fabiochiu/medium-articles huggingface dataset:
Bigfoot Pivot Table & Midterm Ideas
By: Anthony Katok
Tags: Dat, Excel, Statistics

Sorry for the terrible formatting, I had to transfer the pivot table onto here by taking 7 different screenshots from Excel.

Ideas for Midterm

So I’m thinking for this idea would be to incorporate disability statistics comparing trends over time of screening, testing or diagnosis of certain conditions, and comparing that to trends over time of prevalence of given condition. It’s interesting to question whether or not certain conditions in our society are actually increasing in the population or if awareness and testing of the condition has increased and as a result we think the former of the two is the case. The data I likely will be using is from data.census.gov | here’s the link to the database: https://data.census.gov/cedsci/table?q=adhd&t=Disability&tid=ACSST1Y2019.S1810&hidePreview=true

2. For my second idea I want to analyze the country’s mental health, specifically with a focus on Anxiety and Depression, over the course of this pandemic. This data set from the CDC: https://www.cdc.gov/nchs/covid19/pulse/mental-health.htm shows an extensive survey that analyzes symptoms of both anxiety disorder and depression every week since April of 2020. I think I will be eventually comparing this with data generally showing the severity of the pandemic as it’s taken course over the past year, such as number of cases.
Excel Dashboard-Marketing
By: Andrea Allen
Tags: Marketing, Excel, Data Analysis

This dashboard was created from marketing data in Maven data playground. I cleaned the data in power query and used pivot tables to analyze the data. The company objective is to increase campaign response. My thought was to focus on data from customers who had purchased products in the last 30 days. Since these customers are already actively buying, they would be “easy” targets to gain initial momentum. Also, if campaigns are successful with this group, they can evangelize and assist in bringing new customers.

The customers are segmented into active loyal (dark blue) and active opportunity (lighter blue). The dark blue customers stand out largely. Knowing the demographic information and sales percentages these customers bring in can assist the company in creating a loyalty program for this group. The goal should be to maintain loyalty and show customer appreciation. Something along rewards points that build up to receive free gifts or money off.

The active opportunity customers buy but not as much as the loyal group. The company should encourage to buy more with special promotion on favored products like Wine and Meats. Think “Buy 3 bottles of wine get 10% of total purchase”.

This isn’t perfect and will be edited so that the dashboard can be seen better. Or a link offered for a better experience. The main point of this post was for me to practice analytical thinking, creating a dashboard, and presenting my thinking. I welcome constructive criticism.
Microsoft’s Gemini Project and the Excel add-ons that birthed Power BI
By: Yousaf Khan
Tags: Microsoft, Data Science, Excel, Power Bi

In 2009, Gemini was renamed to Power Pivot and released as free excel add-on from Microsoft.

In 2010, Power Pivot gained traction with the help of Rob Collie’s PowerPivotPro.com blog and Excel business users. Power Pivot provided additional data modeling capabilities and ability to mash up large volumes of data. As it gained popularity, Microsoft took notice and invested more into BI toolset.

In 2012, Power View was released as part of SharePoint. It was Microsoft’s attempt to deliver amazing report visuals in the browser similar to Tableau.

In early 2013, Data Explorer was released as an Excel add-on and later renamed to Power Query. Power Query is a data preparation engine that helps you transform and clean your data as well as ingest different sources of data as queries.

Having Power Query and Power Pivot available as Excel add-ons made them accessible but it caused problems when sharing files with lots of data. It was possible to install Power Pivot together with Microsoft SharePoint and create a central place for Power Pivot to work and refresh data. However, with growing datasets and additional requirements, it was getting cumbersome.

In January 2015, Microsoft announced a new Power BI Service that could be accessed at PowerBI.com. Although in preview mode only, over 500k individuals jumped onboard to test it and the team employed a rapid development cycle to release new features fast.

At the time, the product was called Power BI Designer Preview. The application combined Power Pivot Data Model, Power Query and Power View so users can build reports offline and upload to the service.

Power BI Designer Preview (2014)

In July 2015, it exited the “Preview” phase and entered “General Availibility”. Those were some seriously fast sprints!

Today, the Power BI team still continues to deliver monthly releases as part of their overall roadmap and enhance the experience for business analysts. In addition, they keep members up to date and involved within the Power BI community forums.
[Python]A small parser
By: Sue Chi
Tags: Json, Excel, Python

If you ask me what an engineer does? Writing C++? Writing Java? Writing Python?

I’ll say an engineer is who can solve the question in front of herself/himself.

And recently, I face a challenge that I need to parse a long long word file table into my customized JSON file.

SO, it looks reasonable to write some python to help me deal with this problem.

Input：A long long word file table.

Output：A customized JSON file.

Python version：2.7

And the best way for python to access the table data would be excel, so my first step is to paste the table to the excel and clean the unnecessary part(like some duplicate title etc.)

These two imports for accessing excel file:

import xlrd

import xlwt

These two for JSON format:

import json

import collections

And this one is because I need to deal with some Latin language, so I’m gonna need this to assign the decoding and encoding:

import io

Usage of Collection：

len() & zfill()：Can help you to fill some 0’s with specific lengh

The encoding for Latin words：
Masako Wakamiya Workshops on Excel Art
By: Vivita Estonia, Vivistop Telliskivi
Tags: Excel

On June 19–20 VIVISTOP Telliskivi was proud to host Excel-Art workshops by 84-year-old Masako Wakamiya from Japan. Almost 100 people participated in the workshops and morning lecture during these inspiring days.

Masako Wakamiya / photo: Sigrid Kägi

Masako Wakamiya is the eldest programmer in Japan. She creates iPhone apps for seniors and is dedicated to teach and inspire seniors to enhance their digital skills all around the world. After her retirement from the corporate world at the age of 59, she innovated her life through the usage of a computer which supported her relationship with the outside world.

Masako Wakamiya and the participants of the workshop Anne, Külli and Kadri / photo: Marju Kask

Alongside with kids also many bold and beautiful grandmothers joined the workshops on both days and it resulted in a wonderful energy beyond different generations. This week one of the grandmothers is already running her first own workshop for kids at VIVISTOP Telliskivi together with her granddaughter.

Masako Wakamiya is the innovator of turning the world`s most commonly used spreadsheet application — Microsoft Excel — into a tool to enjoy art with a computer. Designs created in the workshop were printed to decorate a unique “uchiwa” — Japanese paper fan — practical accessory for hot summer weather. If you’d want to test this out at home, follow the instructions here.

Workshop participants with Masako Wakamiya / photo credits Sigrid Kägi

Thank you: Robotex Japan, Embassy of Japan in Estonia, Microsoft Estonia, Janne Funk (OÜ Kirimiri)

From left: Masako Wakamiya, Janne Funk, Eva Liisa Kubinyi, Halina Mugame, Marju Kask, Mari-Liis Lind, Yuriko Saito and Takumi Amano / photo: Sigrid Kägi

Watch Masako Wakamiya’s TEDx Tokyo talk here.
-> Microsoft Excel 2013: Data Analysis and Business Modeling by Wayne L. Winston -> Available in Hardcover \ Kindle \ Paperback \ AudioBook
By: Namin Madridi
Tags: Excel

-> Microsoft Excel 2013: Data Analysis and Business Modeling by Wayne L. Winston -> Available in Hardcover \ Kindle \ Paperback \ AudioBook Namin Madridi Sep 9, 2019·1 min read

Just For Today get free read 30 days !!!

Master business modeling and analysis techniques with Microsoft Excel 2013, and transform data into bottom-line results. Written by award-winning educator Wayne Winston, this hands-on, scenario-focused guide shows you how to use the latest Excel tools to integrate data from multiple tables?and how to effectively build a relational data source inside an Excel workbook.Solve real business problems with Excel?and sharpen your edgeSummarize data with PivotTables and Descriptive StatisticsExplore new trends in predictive and prescriptive analyticsUse Excel Trend Curves, multiple regression, and exponential smoothingMaster advanced Excel functions such as OFFSET and INDIRECTDelve into key financial, statistical, and time functionsMake your charts more effective with the Power View toolTame complex optimization problems with Excel SolverRun Monte Carlo simulations on stock prices and bidding modelsApply important modeling tools such as the Inquire add-in … (More info! -> https://onlinebookreaderread.blogspot.com/?book=0735669139)
Populate PDF forms with Excel data
Tags: Data Merge, Excel, Pdf

DocuClipper makes it easy to quickly create documents by automatically populating richly-formatted templates with data from Excel.

Common use cases include populating tax forms, invoices, and reports.

Watch this video on how to create PDF documents with data from Excel or keep reading below.

Creating a template

The first step in the process is to create a template that specifies where to put the data.

To create each field, click and drag to draw a box around the date. Fields can be of type “Text” or type “Checkbox”.

Creating the Excel spreadsheet with the data

Once we have a template, we can generate multiple customized documents with data from Excel. You can download the appropriate spreadsheet template with the column names already filled in so you only have to transfer over the values.

Each field that was created in the template has its own column.

DocuClipper will create an individual PDF document for every row in the spreadsheet.

Downloading the created documents

It only takes a few seconds to generate the documents. Once the process is done, click on “Download” to get the documents.
Automate Your Simple Reporting with R
By: Putik Dhiraramanti
Tags: Reporting, Excel, R, Analysis

Photo by Thomas Tastet on Unsplash

As an analyst, do you feel annoyed when you get a report request every single day? Your organization is not there yet with an integrated reporting system like Tableu, PowerBI, and Qlikview. You are stuck with getting the report from the database and sending it as an Excel file.

Worry no more. In this article, I will share a simple step-by-step process to automate basic reporting using R. If you do not have R and RStudio already installed on your machine, you should install them firsthand.

Download and install R from this page https://cran.r-project.org/. Download the R IDE; for me, I use RStudio https://rstudio.com/products/rstudio/download/. The library that we will use in this project is sqldf, RMySQL, and mailR. Make sure you have them load in your project, you can use install.packages() function.

I use this sample data on this article, and the data was imported to the MySQL database in my local machine. First step is to load the necessary packages on your project.

library(sqldf)

library(RMySQL)

library(mailR)

Next step will be to load the data that you need to show. In this case, the example will be how many units of products were sold yesterday.

Next step will be showing the content of the email that you want to send.

The final step will be sending the mail using mailR.

And this is what it looks like on the mail that will be received.

This is a simple functionality that you can start with. For nicer looks of email rendering, you can use Rmarkdown to create better looks of the mail by creating html file of the mail that you want to send. Approximately, it will look like something like this below.

Enjoy the exploration!
5 Tools Every Data Scientist Should Use
By: Jake Mito
Tags: Pandas, Data Science, Excel, Python

Pandas

I know this is obvious, but Pandas is a really amazing package for run of the mill data science, but it also does some pretty complex things many people do not know about.

For example, in Pandas you can do web-scraping with the read_html function:

pd.read_html("url")

Or find the counts of all the unique values in a data frame:

df.nunique()

2. Mito

Mito is a spreadsheet that generates Python. You get to feel like you are working in Excel or Google Sheets and each edit you make generates the equivalent Python.

In Mito, you can:

Merge

Filter

Pivot

Graph

Use Formulas

and much more!

Each edit will generate the equivalent Python in the code cell below.

Here are the Mito install instructions

Here is a demo video:

3. Tensorflow

Tensorflow is a ML package that allows users to create models very quickly. It was originally developed at Google. Tensorflow has become a popular first package for those beginning to engage with Machine learning.

Within the package, you can do things like:

Build neural networks

Design recommendation systems

Create adversarial networks

They also have a wealth of server partners that help host these models. These representatives can help set up the cloud/local environments, as well as provide consulting on how to structure them.

Here is a great 10 minute introduction video to TensorFlow.

4. Beautiful Soup

Beautiful soup is a Python package that allows for easy data extraction from html and xml formats — this essentially means web-scraping.

To install:

$ pip install beautifulsoup4

from bs4 import BeautifulSoup

soup = BeautifulSoup(html_doc, 'html.parser')

The package has easy commands to parse through the html.

To retrieve the title:

soup.title

To find all the URLs in the page:

for link in soup.find_all('a'):

To retrieve all the text on a page:

print(soup.get_text())

Here is the full documentation.

5. Matplotlib

Matplotlib is one of Python’s best graphing packages. You can make simple visual representations as well as more advanced, interactive graphics.

These are the install commands:

python -m pip install -U pip

python -m pip install -U matplotlib

Here is the code for a simple line plot:

There are many other more advanced visualization options as well.

Stacked Bar Chart

Filled Polygon:

Here is the full documentation

I hope you found these tools useful :)
Beginner’s Guide to exceljs
By: Ng Wai Foong
Tags: Nodejs, Web Development, Excel, Programming, JavaScript

Setup

The recommended installation process is via npm which mainly works for node.js project. If you intend to use it directly within a browser for non-node.js project, you have to import the package using the official CDN link. Let’s have a look on the installation process for both methods.

npm

Run the following command to install it in your project:

npm install exceljs

Then, you can simply import it as follows:

const ExcelJS = require('exceljs');

For node.js that is older than 10, you have to import using ES5 transpiled code. Also, you need to import a few polyfills as well:

// polyfills required by exceljs

require('core-js/modules/es.promise');

require('core-js/modules/es.string.includes');

require('core-js/modules/es.object.assign');

require('core-js/modules/es.object.keys');

require('core-js/modules/es.symbol');

require('core-js/modules/es.symbol.async-iterator');

require('regenerator-runtime/runtime'); const ExcelJS = require('exceljs/dist/es5');

CDN

For using it directly on your browser, simply head over to the official CDN page for exceljs and copy the corresponding script tag based on your needs:

For example, you can use the following script tag inside your HTML file:

(Optional) FileSaver.js

For your information, some of the features in exceljs are highly dependent on node.js built-in packages. For example, the save file functionality requires the fs module. This will throw an error when you are calling it within non-node.js project.

Having said that, you can still achieve file saving feature with just a few additional tricks. All you need to do is to output the binary data as buffer and save it using an external JavaScript library such as FileSaver.js. Head over to the following repository and copy the script tag based on your use cases.

Then, simply insert the script tag in your HTML as follows:
The most overlooked opportunity for Excel users is visual design
By: Josh Cottrell-Schloemer
Tags: Spreadsheets, Data, Design, Excel, Model

When you think of Excel, you probably think of spreadsheets with rows and rows of data. Maybe you imagine a few fields for inputs and outputs. If you’re feeling fancy, you might even have a couple charts in there.

Something like this:

It’s functional. It shows your metrics. But it’s nothing particularly exciting.

What a lot of Excel users don’t realize is that with a bit of finagling and a basic understanding of Excel’s built-in design features, you can have something that looks a lot more like a custom dashboard:

This one is built using Excel’s standard features (no fancy add-ins or scripting). It uses the exact same set of data used in the “before” pic. But it has a very different impact on the user.

Now this isn’t a class in building dashboards in Excel but I still want to illustrate how many visual design features are available and remain unused by 98% of Excel users.

Obviously not everyone has a design background but almost everyone can improve the visual design of their Excel sheets simply by getting familiarized with the tool’s design features — the short version is that almost anything you do in Powerpoint can be done in Excel. You can insert shapes and images, use gradients and transparent layers, customize charts and so much more.

In the wild world of spreadsheets, it’s easy to fall into the trap of thinking functionality is the only thing that matters. But humans are irrational creatures and will judge a book by its cover. Spending a few extra hours to polish up the design of your spreadsheet can be a big factor in your project’s success.

If you’re making a living using Excel then this becomes even more important. One of the easiest ways to differentiate your work is to add an element of visual design. 90% of your competition is probably using the same basic style and layout with zero visual impact. With just a bit of time dedicated to visual design you can instantly stand out from the crowd and give your work a feeling of credibility and customization. On many projects that means changing your scope by <5% and will likely have a disproportionately big impact on your business.

This is one of those low barrier opportunities that gets overlooked far too often. Take the time to learn about these design features in Excel or have a designer on your team get familiar with them. Add an extra hour or two when you’re estimating scope for your next project. Then deliver something that looks great. If you notice that folks get more excited about the spreadsheets you build then do it again. My guess is that you’ll be pleasantly surprised by the results.
How to Send and Receive Automated Emails Using Python
By: M Khorasani
Tags: Dashboard, Data Science, Data Visualization, Email, Excel

How to Send and Receive Automated Emails Using Python

Introduction

We send and receive a plethora of emails on a daily basis in our corporate and non-corporate lives. And truth be told, a large portion of those transactions can indeed be automated. Perhaps we can all relate to a landmark moment of our lives when we spent the good portion of a day sending more or less the same email to a multitude of people. Well, it doesn't have to be that way. With Python, you can in fact set up and deploy an automated email pipeline to receive emails and subsequently send emails based on the content of the former. In other words, you can put automation to work while you attend to more impactful tasks at work or home.

There is absolutely no need to be tech-savvy and what we are going to do is trivial enough that each and every one of us can do so in short order. After this tutorial, you will learn how to automatically receive emails, use the content and attachments to generate charts, and then finally send emails with dazzling dashboards to a number of recipients.

Specifically, in this tutorial we will automate the following actions:
Apache Superset: Export Data to Excel
By: Aleksandr Gordienko
Tags: Export Data, Frontend, Apache Superset, Excel, Docker

Apache Superset 1.1.0 with ‘Export to XLSX’ functionality enabled

Apache Superset is a modern BI tool that helps you easily create rich charts and powerful dashboards by combining information on dashboards from various databases connected via SQLAlchemy. Gathered from different sources into a chart, data can be simply exported then to a comma-separated file (CSV) from the Superset right out of the box. That CSV file could be opened via many spreadsheet software including industry-leading Microsoft Excel. But there is at least one problem that could bring you to the point at which you will want to patch your Superset installation to be able to export data from charts to the XLSX format. And that problem is Microsoft Excel’s auto-recognition of values as dates.

Let’s imagine you have a simple table exported from Superset to a CSV file. Here is its content:

Name,Building,Street,ID

Liubov,23/1,Pretty St.,01-04

Mike,17,New St.,16-7

And here is how you would probably think it would look in your spreadsheet:

┌────────┬──────────┬────────────┬───────┐

│ Name │ Building │ Street │ ID │

├────────┼──────────┼────────────┼───────┤

│ Liubov │ 23/1 │ Pretty St. │ 01-04 │

│ Mike │ 17 │ New St. │ 16-7 │

└────────┴──────────┴────────────┴───────┘

If you will try to import this CSV file into Excel, you will be surprised. It is not easy to find the table’s initial view correctly without additional user actions. Your table will probably look like this by default:

┌────────┬──────────┬────────────┬────────┐

│ Name │ Building │ Street │ ID │

├────────┼──────────┼────────────┼────────┤

│ Liubov │ 23-Jan │ Pretty St. │ 01-Apr │

│ Mike │ 17 │ New St. │ 16-Jul │

└────────┴──────────┴────────────┴────────┘

As you can see, Excel is trying to interpret some numbers as dates where it’s possible based on your system's regional settings. And this problem could lead to errors in decisions people make on the data they see in the table.

There are few ways how to fix this. You could teach your users to open CSV files with additional actions: choosing an ‘Import’ menu item, splitting tables into columns, setting the data type for each column, and so on. This could prevent Excel from auto-interpreting numbers as dates and using user-selected data types. Another solution is switching to different spreadsheet software, Only Office for example, which will ask how to interpret data in CSV file at the opening. But I have found all the options are not good enough as they lead to additional costs for the user consultations or changes in the software list you support in your company (and maybe additional license payments). Therefore, I went through a different pathway.
R_Note01 Transform Date Column Into Tidy Data
By: Mr Ding
Tags: Tidyverse, Data Science, Excel, Lubridate

Here is a short note recording how I deal with excel data:

What will happen when the date is the column names of an excel table and got read into R using read_excel (Default Rstudio functions)?

A: the colomn name will be read in as a character.

library(lubridate) #date convert

library(readxl) #read in excel

library(stringr) #string transform

library(tidyverse) df <- read_excel("date.csv") names(df) [1] "Region" "18年1月" "18年2月" "18年3月" "18年4月" "18年5月" "18年6月" "18年7月" [9] "18年8月" "18年9月" "18年10月" "18年11月" "18年12月" "19年1月" "19年2月" "19年3月" [17] "19年4月" "19年5月" "19年6月"

Note: I keep the data as it is. So it is in Chinese. the format is 2d%year2%month in Chinese

So here the regular steps to tidy up this:

Use Gather function change the string to a format that a date converting function can understand Convert to a datetime object



df %>% #step1 gather(key="time", value="KPI", -c("Region"), na.rm = TRUE, factor_key = FALSE) %>% #step2 with str_replace then step3 with ymd mutate(time = str_replace(time,"年","-") %>% str_replace("月","-01") %>% ymd() )

But this is not my first approach, I was trying to convert column name from character to date:

#Step1 Create Date m_list<- seq(lubridate::ymd('2018/01/01'), lubridate::ymd('2019/06/06'), by='month', truncate=2) #Step2 update column name colnames(df)[2:19] <- m_list colnames(df) [1] "Region" "17532" "17563" "17591" "17622" "17652" "17683" "17713" "17744" "17775" "17805" [12] "17836" "17866" "17897" "17928" "17956" "17987" "18017" "18048"

We will see that the column will convert into numbers, not date. ! The numeric is actually a number reflect the number of days since “1970–01–01”. To read more check out the document about lubricate

#step 3 df %>% gather(key="time", value="KPI", -c("Region"), na.rm = TRUE, factor_key = FALSE) %>% mutate(time = time %>% as.numeric %>% lubridate::as_date)

Well, this finally works.
HOW ESTIMATING THE STOCK VALUE BY EXCEL
Tags: Stock Market, Estimations, Excel, Finance, Share

As you can see, with the timeline I estimated the ENI’s Value per each year. With all available data information, I could calculate the Future Value, known also as the Face Value, of this Stock per every single year by an easy Excel Formula:

=FV(rate,nper,pmt,pv)

This is the easiest and fastest way to calculate the Capitalization of a certain stock during a certain period of time. The rate is the Current Yield, which is equal to 1.07%. Following this, the nper is just the investment period. While the pmt is the rate to pay every single period, such as a month, year, … Finally, the pv is the Present Value which in our case is equal to 13,380.00 euro (including the bought 1,000 shares).

In addition, once the FV of every single year is calculated, it would be possible to get the Coupon, which is the possible income for holding the stock until its maturity. Indeed, we can get it by multiplying the Face Value per Coupon Rate.

Finally, by following the last chart you would see that if I buy now ENI and I would hold it for 12 years I would get more or less 910.00 euro per each year as a possible income for this kind of investment.

Consequently, it would be possible to calculate the potential Net Income at the maturity of this stock:

As you can see, in this final chart I firstly summed all coupons during these 12 years. Following this, I calculated the Capital Gain, which is the difference between the ENI’s Present Value and its Future Value. Thus, I could get the Total Income which is 10,330.62 + 1,822.79. Then we must consider taxes. In my country, Italy, the Tax Rate on Income is 26%. Therefore, the Net Income is 8,993.52 euros. So, the annual ROI is at around 6%, which is given by dividing the Net Income by the Total Investment and so, in this case, 13,380.00 euro.

In the next blog, I would explain to you how it would be possible to value the portfolio performance always by using Excel.

Really thanks and enjoy the reading.

See you ;)

DATA LINKS
Getting the Last Row with Values on Google sheets
Tags: Spreadsheets, Google Sheets, Google, Excel, Programming

Recently at work, I had assisted a colleague solve a seemingly complex problem on google sheets. The solution worked “perfectly”, but there was a small issue ; The range doesn’t automatically increase when a new value is added to a row. So the solution involved this colleague of mine having to go back to the formula to increase the range whenever a new value was added. This didn’t seem like such a perfect solution so I needed to come up with ways to solve this. I know someone might be yelling by now saying “why don’t you just do A:A as range”. The thing with this problem is that doing A:A will count blank rows which will pollute our result.

The quest for a solution led me to learning some really nice and handy functions and I want to share what I learnt.

I discovered two ways to solve this particular problem. For the purpose of this tutorial we will try to calculate the sum of every range that has a value in column A

=sum(A:index(a:a, max(a:a <> “”))) : This formula first checks the maximum value that has its value not equals to an empty string. In simple terms, it checks which row has a value, the index formula returns content of a specified cell. This is how the index formula works : =index(cell_reference, [row], [column]) . The cell reference is the range that has the data with the contents we want. The row argument is optional but if we wanted to return a particular value from a particular row to be returned, we will specify the row number. In this case, we used the max formula to get the row number that had a value and makes it the second argument in the formula. This formula will successfully increase its range to sum everything in cells that have a value in them. Another interesting formula to use is =sum(max(arrayformula(if(A1:A <> “”,row(a:a),”” )))). We use an if statement to check if a1:a is not equals to an empty row then return the row number, else return an empty string. We are able to return the row number because we are using the arrayformula. The Row() formula only works well with range if we use the arrayformula else it will return the row number of the first value in the range . For example row(a2:a9) without the arrayformula will return 2. The max formula is why we can now return the value.

What if we wanted to return the content of the last row instead of just the cell number? We can tweak the second formula a little bit and add the indirect() formula. Indirect returns the content of a cell reference specified as a string. =max(arrayformula(if(a1:a<>””, indirect(“a1:a”),””) This will return the value in the last row. We can replace max with min and return the content of the first row with a value.

I hope you found these formulas helpful. If you enjoyed reading this please leave a clap.

You can connect with me on twitter .

Thanks for reading.
Create Filter in Excel using Java
By: Andrew Wilson
Tags: Filters, Excel, Java

When dealing with Excel documents containing a large amount of data, you can select the data you need quickly by filtering the data. This article will demonstrate how to create filter in an Excel file using Free Spire.XLS for Java.

Installation (2 Method)

1# Download the free API and unzip it, then add the Spire.Xls.jar file to your project as dependency.

2# You can also add the jar dependency to maven project by adding the following configurations to the pom.xml.

<repositories>

<repository>

<id>com.e-iceblue</id>

<name>e-iceblue</name>

<url>http://repo.e-iceblue.com/nexus/content/groups/public/</url>

</repository>

</repositories>

<dependencies>

<dependency>

<groupId>e-iceblue</groupId>

<artifactId>spire.xls.free</artifactId>

<version>3.9.1</version>

</dependency>

</dependencies>

Code Snippet

import com.spire.xls.ExcelVersion;

import com.spire.xls.Workbook;

import com.spire.xls.Worksheet;



public class CreateFilter {

public static void main(String[] args){

//Load an Excel file

Workbook workbook = new Workbook();

workbook.loadFromFile("sample3.xlsx");



//Get the first worksheet

Worksheet sheet = workbook.getWorksheets().get(0);



//Create filter

sheet.getAutoFilters().setRange(sheet.getCellRange("A1:F1"));



//Save the result file

workbook.saveToFile("CreateFilter.xlsx", ExcelVersion.Version2010);

}

}

Output
From Excel to Python: How to do COUNTIFS with Pandas
By: Alex Mealing
Tags: Spreadsheets, Excel, Python, Data Analysis

Photo by Lukas Blazek on Unsplash

COUNTIF is an essential spreadsheet formula that most Excel users will be familiar with. I’ll now show you how to achieve the same results using Python (specifically the pandas module).

Here is a dataframe that contains a list of all the athletes that have competed in the olympic games since 1896. We want to know how many athletes each country has fielded for all of these games. The data looks like this:

Using Excel, we would need to create new rows with each of the country codes, and then write a formula that looks something like =SUMIF(name_column, country, country_range)

With pandas, we use the “groupby” formula to do the hard work for us:

How does this work?

athletesDf.groupby([‘noc’])… -> Tell pandas to create “groups” based on the ‘noc’ column for the dataframe “athletesDf”. This doesn’t do anything yet.

…[[‘name’]].count() -> Tell pandas to count all the rows in the spreadsheet. It doesn’t really matter what column we use here because we are just counting the rows

Let’s now add another variable to turn the COUNTIF into a COUNTIFS:

That was easy. All we did was add another column in the groupby bracket to say how we wanted to split the data again -> athletesDf.groupby([‘noc’, ‘sex’])…

Thanks for reading! Stay in touch if you are looking to develop your data analysis skills and enjoy these quick tips!

-Alex
Day 6: The Difference Between Two Spreadsheets
By: Gemma Black
Tags: Full Stack Developer, Excel, Vlookup

Day 6: The Difference Between Two Spreadsheets

Yesterday, I had a simple request. What data items are in one spreadsheet and not in the other?

I attempted to import the CSV files into MySql to run a query.

I got import errors.

So I began to wonder, “maybe I’m overthinking things…again”. Each file was only 11,000–12,000 lines long.

So here’s what I did.

Step 1: A line count check

Both files had the same name.

wc -l file1.csv && wc -l file1 (1).csv

Both had the same number of lines.

Step 2: A diff check

diff file1.csv file1 (1).csv

No difference.

Job done! No difference.

Except.

I was comparing the wrong two files.

Next. I got hold of the two actual files for comparison.

I ran the wc and diff command again. Now there were 2000+ differences.

At this point, my Linux skills are lacking, and so I turned to Excel to get the specific differences from one file, not across both.

Step 3: Excel-ing it

Here are the steps:

Open up Excel

Import each csv file into separate tabs

To find missing data in the other spreadsheet, add the command below to an empty field on the same row.

=VLOOKUP(A1,SheetName!$A:$A,1,FALSE)

A1 represents the cell you want to check.

SheetName!$A:$A is the sheet and the range you want to check against.

1 is the index if the range spans multiple rows. I left it at one.

FALSE is for an exact match check.

Copy that command down all the rows. And you’ll find that, where there’s no match, N/A will come up.

will come up. And voila! You’ve found all data items that are in one spreadsheet and not in another.

Lesson: many solutions don’t require scripting

This is not a very exciting article, but it does highlight, that as a developer, my goto tools are code and databases. Sometimes, there are better tools that require no code.
Practicing Data Visualization with the North American Breeding Bird Survey
By: Raina Sciocchetti
Tags: California, Tableau, Data Visualization, Birds, Excel

Data visualization is everywhere. Beautiful and coherent charts, graphs and fancy little graphics transform raw numbers into entire stories.

When it comes to data visualization, I could be more clueless, but I’m not exactly a master either. My go-to technique is to bludgeon data into submitting to my will, which usually requires a great deal of trial or error and compensating for my lack of skills with dogged manual labor as necessary.

This project proved no different. With an impressive “looking for data” to “actually manipulating data” ratio of 15:1, I bludgeoned, trialed, mostly erred and compensated my way through a faulty dataset and a couple of different data visualization platforms to arrive here and with — even more impressively, I think — something to show.

Selecting a Data Set

Every good data visualization starts with a dataset. I landed upon bird population data from the Migratory Bird Data Center, a partnership between the U.S. Fish and Wildlife Service, Division of Migratory Bird Management and U.S. Geological Survey, Patuxent Wildlife Research Center.

Next, I selected a raw data set from the North American Breeding Bird Survey, a long-term, large-scale avian monitoring program that is used to estimate population trends at various geographic scales. My customized data set described a species totals report for both breeders and migrant or non-breeder species reported at Fish Rock, CA between 1968 and 2014.

Customizing a data set using the online retrieval tool.

The Data Visualization Begins

In order to facilitate learning and experimentation, I decided to select just eleven common birds from the data set on which to practice.

Fish Rock is a coastal location in Mendocino County in Northern California.

Population trends of 11 common avian breeders at Fish Rock, CA between the years of 1968 and 2014. Created with Excel.

The following slides show the distributions of the selected eleven avian breeders species counted at Fish Rock, CA in 1970, 1982, 1992, 2002 and 2012. Or, in other words, which birds are more common, and how did their numbers in respect to each other change over each decade? Created with Tableau.
Scope of Financial Modeling & Valuation in 21st Century!
Tags: Cfa, Excel, Financial Modelling, Acca

Financial modelling & valuation has begun to emerge as a critical method in today's data-driven world, where most decisions are highly quantitative. It has become the most sought-after skill needed in almost all components of the financial services industry, such as banking and equity analysis. It is an ideally crafted course aimed at providing realistic exposure to individuals who believe in long-term investment and value development. This curriculum is structured in such a way as to help participants not only understand the very essence of the topic but also apply that understanding to the creation of practical and consistent financial models for real-life problems.

What AR Finance room has to offer:

For professionals with full-time jobs, this education company provides well-organized sessions through E-learning. The education programs aim to train learners to gain in-demand skills, extract insights, and improve decision making needed to succeed as financial analysts. The courses are structured by experts to include hands-on knowledge and experience needed to excel in the field.

Program Features:

To be able to create a cash flow statement

Learn how to construct a valuation model from scratch

Understand how an organisation is priced

Learn how to build a multi-scenario model

Learn how to measure the value of the company and the value of equity

Develop the skills of Microsoft Excel modelling

To be able to do sensitivity analysis

Practice predicting skills

Creation of the Financial Model prototype and Excel tips

Fletting out a dedicated Model Input sheet to compile all the client's plans

Performing the DCF Valuation

Computing the financial ratio

Learn how to construct a financial model from scratch

Understand how to calculate discounted cash flow

Develop financial models for start-ups and existing firms

Learn how to build stable, scalable and real-world financial model models

The methodologies and strategies in the finance industry are also going forward with time and new technologies. This led to the idea of financial models that aid in efficient and simple research, making financial modelling a prerequisite for finance professionals in India.

Job Opportunities:
Bridging Excel with Python: xlwings
By: Ernest Huang
Tags: Simulation, Python, Excel, Stochastic Modelling, Xlwing

Recently, I did several oil and gas commodity price simulations using stochastic techniques to evaluate the risk exposure of a regional oil and gas player. These simulations involved calculating 10,000 sets of prices x 5 price indices, on a monthly level for 3 years.

The traditional approach would have been to use a 3rd party Microsoft Excel plugin to perform these stochastic simulations, but from some research and other’s past experiences, I found that:

It cost about 3,000 USD per user per year for a license 10,000 simulations would take hours to run on MS Excel, and would crash frequently It had inflexible functions, and was hard to customize to the client’s situation

Personally, I feel MS Excel is always good for fast data wrangling with little set-up and is good for visualizing the flow of calculations, but generally suffers under heavy computing.

Another option was to implement the stochastic simulation in Python. This solved many of the problems highlighted above:

Absolutely free, with dozens of open source packages for stochastic simulation, optimization, etc. The Pandas package in Python is suited for heavy computing: 10,000 simulations took minutes to run, compared to hours Fully customizable functions and simulations

However, there was an issue…

The client was much more comfortable using Excel than Python/Jupyter notebooks. So how would we be able to successfully handover our Python based simulation tool and deliver lasting impact?

Enter xlwings

I’m surprised this package gets so little hype for what it’s able to do.

xlwings acts as an interface between a Python kernel and Excel workbook. It’s able to:

Read specific data from an Excel workbook into memory, and print data, graphs and pictures generated by Python into specified places in an Excel workbook Use the in-built Excel ribbon to import your Python functions that can be used in the Excel formula bar or as VBA macros (The VBA code is auto-generated for you!)

With these features, I built a bespoke input Excel workbook, with all the explanatory notes and formatting required. xlwings was used to link the workbook to Python functions in a separate script, and the client could realize all the benefits of Python without mucking with Jupyter notebook, nor writing a single line of code.

Illustrative workflow for our commodity price simulation tool

After some painless installation of Python and the needed packages, the client took to the workbook quickly. They particularly enjoyed clicking on the macro buttons and seeing the results printed to the Excel book instantly (“What if I change it to 50%? To 100%? Let’s compare the results now”). I could see that this format greatly encouraged experimenting around with different strategies.

As consulting starts to rely more on advanced analytics like machine learning, big data visualization and scenario simulations, I predict xlwings will quickly become a favorite tool in my toolkit.

Go check out xlwings here: https://docs.xlwings.org/en/stable/ ✌
Azure WebApp to read Excel files without having Excel installed
By: Ahmed Adly
Tags: Web App Development, Microsoft, Documentformat, Excel

Once, I was requested to create an Azure WebApp that reads from Excel files, It will be SaaS, and this means it should be independent on Excel Application, and to read rows and columns of Excel without the existence of Microsoft Office Excel binaries.

Luckily, there is a way to read from Excel files that has (*.xlsx) or (*.xls) extensions, without having Office Excel installed on your machine, using an open source library.

This way can be used with any .NET project type (Windows, or Webs), also can be used with PowerShell, however, I will address below how to use it with .NET only in details, and will give some guidance how to use it with PowerShell

Open your visual studio, and Create a new .NET Project (web app, or windows app, or other) with .NET v4.0 or more, I will choose the name “AppReader” we need to add an open source library, To install it you need to open “Package Manage Console” in visual studio

3. Then run the below command in the console :

Install-Package DocumentFormat.OpenXml -Version 2.9.1

If you prefer to do it manually, you can search google on DocumentFormat.OpenXml, download the .dll, and use add to reference

4. Now, Create a new Class, and lets name it “ExcelAdapter.cs”

5. Paste the below block in “ExcelAdapter.cs”

6. Now the Document Format should be installed in your project, check the references (or bin) to confirm

7. Rebuild the project, should be no errors

8. To start using the functions create a new Webform, or Windows Form, and start using the functions, this function will return list of sheets in the excel file

List<string> sheets = ExcelAdapter.GetSheet(@"~Path\file.xlsx");

9. Now we will get the data in one of the sheets, and load it to a datatable, then to DataGrid, to do this write the below

DataTable dt = ExcelAdapter.ReadExcel(@"Path\Sales.xlsx", "Sheet1");

this.GridView1.DataSource = dt;

this.GridView1.DataBind();

//Please don't forget to add databind if you are using with asp.net

10. and you will have your Dataload as below

If you want to use it with PowerShell, it will be a little bit more complicated, you will need first to, convert this module to *.dll then reference it to your PowerShell script like in this link

https://activedirectoryfaq.com/2016/01/use-net-code-c-and-dlls-in-powershell/

Regards,
How to save JSON data in EXCEL file using Node.js
By: Shadab Shaikh
Tags: Nodejs, NPM, Excel, Technology, JavaScript

Sometimes, you have a 3rd party API which returns data in JSON format and you need that data in excel file. How we can do this? Let’s follow the steps:

We will save our JSON data in the EXCEL file using an excel4node library in Node.js.

Let’s jump to the code and then I will explain code line by line.

Create index.js

Create package.json using

npm init

Install excel4node using

npm install — save excel4node

Define your data you want to be store in excel

const data = [

{

"name":"Shadab Shaikh",

"email":"shadab@gmail.com",

"mobile":"1234567890"

}

]

Import excel4node library

const xl = require('excel4node');

Create a workbook and give some awesome name

const wb = new xl.Workbook();

const ws = wb.addWorksheet('Worksheet Name');

Now Let’s define columnName

const headingColumnNames = [

"Name",

"Email",

"Mobile",

]

Before moving to next let’s explore some functions in excel4node library

1. cell(rownumber,columnnumber)

requires 2 parameters

a. row number(starts from 1)

b. column number(starts from 1)

This function selects cell with given row no. and column no. 2. string(data) , number(data)

we can store data as string or number

just call the above functions and pass data in it.

Now write columnName in Excel file using functions in excel4node

let headingColumnIndex = 1;

headingColumnNames.forEach(heading => {

ws.cell(1, headingColumnIndex++)

.string(heading)

});

Finally, write our data in excel file

(Don’t forget to start row number from 2)

let rowIndex = 2;

data.forEach( record => {

let columnIndex = 1;

Object.keys(record ).forEach(columnName =>{

ws.cell(rowIndex,columnIndex++)

.string(record [columnName])

});

rowIndex++;

});

Now Let’s take workbook and save it into the file

wb.write('filename.xlsx');

Here is full code, just copy and paste in your favorite editor to go through demo.

const xl = require('excel4node');

const wb = new xl.Workbook();

const ws = wb.addWorksheet('Worksheet Name'); const data = [

{

"name":"Shadab Shaikh",

"email":"shadab@gmail.com",

"mobile":"1234567890"

}

] const headingColumnNames = [

"Name",

"Email",

"Mobile",

] //Write Column Title in Excel file

let headingColumnIndex = 1;

headingColumnNames.forEach(heading => {

ws.cell(1, headingColumnIndex++)

.string(heading)

}); //Write Data in Excel file

let rowIndex = 2;

data.forEach( record => {

let columnIndex = 1;

Object.keys(record ).forEach(columnName =>{

ws.cell(rowIndex,columnIndex++)

.string(record [columnName])

});

rowIndex++;

});

wb.write('TeacherData.xlsx');

I hope this was helpful… :-)