fabiochiu/medium-articles huggingface dataset:
We’re looking for a Service Design Lead
By: Giulia Merlo
Tags: Service Design

Join us to lead our Service Design team and practice, and help design services that impact the lives of millions of people.

Take me to the job application [closing date September 15th] >>

Chat with Giulia about the role >>

Who’s in the team?

Together with User Research and Product Design, Service Design is a core discipline within our User Research and Design team. We’re part of Customer Journey, in the wider Design, Data and Technology department.

At the moment, we have a multi-level team of 8 Service Designers as well as an active community of practice.

How do we work?

We work in multidisciplinary teams and within agile frameworks to help design services and products that affect the lives of millions of people.

We often design digital journeys, but we’re not limited to digital. We believe Service Design is about understanding, translating and joining up user and organisational needs into the end-to-end design of — no matter the channel. So our Service Designers work on discoveries, prototypes and continuous improvements that span the breadth of our advice services.

We care deeply about inclusive working practices and diverse teams. If you’d prefer to work flexibly, or part-time, we’ll facilitate this — whether to help you meet other commitments or to help you strike a great work-life balance. Several of our leadership team work 4 days a week, and we all work from home regularly.

What’s the role?

We’re looking for someone with lots of service design experience to lead our Service Design team and practice,

In this role, you’ll develop, own and drive a strategic, inclusive, user-centred approach to Service Design that meets our organisational objectives, and aligns with Citizens Advice’s strategic ambitions.

You will create the right conditions for Service Designers to work effectively, set the direction of their practice, assure the quality of our work, and ensure we always design with all users in mind. You’ll work in close collaboration with user researchers, product and content designers, and teams across Citizens Advice.

Would you like to chat?

My DMs are open on Twitter, and you can email me here if you have any thoughts or questions.

I’m also holding this time for informal,open video calls to answer any questions you may have about the role:
Weeknotes by Chan #08–11
Tags: Self Care, Fitness, Mental Health, Interaction Design, Service Design

01. It’s a wrap!

Filter designs in Figma (Design tool)

Our research rainbow containing some eye-opening insights.

Last week, I jumped off the Mind project after working on two sprints to help the charity ‘Create a feature that will allow users to have more control over their experience and make better connections on Side by Side’. An online community which allows people to share their mental health experiences and connect with others.

9 days, 8 research sessions and 10 prototypes later, we developed one idea that launched for first release last week!. A ‘filter’ helping users to navigate the feed, find content more seamlessly and make a meaningful connection with someone facing similar interests or experiences. We still have a lot to learn and a future of iterations but we’ve made the first step.

I was bought to tears during parts of the project, especially when listening to users open up about their mental health experiences and how important this online community is to them in sharing their emotions and making connections. The team at Mind have been absolute superstars too, always eager to learn our design methods and I am hoping they feel empowered to continue to embed these practices of research and design within their organisation too.

It’s an honour to be able to work on a project like this and seeing our imagination come to life — looking forward to see the impact. Writing a case study on this soon so watch this space! 👀
Service Design 🤓
By: Pedro Gaspar
Tags: Creativity, Customer Service, User Experience, Design, Service Design

Service Design has become something of a hot topic over the past few years, however, does not lend justice to its age. Service Design has existed since early eighties.

But what is Service Design? It’s important to realise services are not tangible goods. An interface is not a service. A product is not a service. Service Design it is about the areas, relationships, connections between and across products.

Service Design is the activity of planning and organising business’s resources in order to improve its quality and interaction between service provider and customers. Ideal services are considered to be user-friendly and competitive within their market.

Principles of Successful Service Design.

Keep it simple stupid.

If you build a simple service, it’s logical that people spend less time figuring out how it works. If things are built with simplicity in mind, they are easy to understand and this clarity will ultimately make people stick to your product. Simple services will also be easier to build, which means you can adapt faster than your competitors.
How to: use your walls to share your work inside your organisation and to the wider world
By: Kirsty Joan Sinclair
Tags: Service Design, Process, Working In The Open, Design

For example : using colour, shape and numbers to help others flow through the work, and add their feedback.

So here’s some lessons I’ve learned.

1. Use the same labels for each of your sections.

Whether it’s a single colour of card or post-it note, with the same size writing, or a print out (like the ones Paul Smith put together as templates in this Google slide deck, alongside this helpful blog post of how to use them.) In fact, many of Paul’s template labels could be useful to you. Use them!

Having a constant visual clue will help people new to your work a. scan how many sections there are, b. see the work flow and progress across the wall.

2. Make those labels easy to understand.

Use language that’s clear, and explain what you are/were trying to do at each stage of the work.

3. Lay out your work chronologically, and put the ‘best bits’ on the wall.

I work (mostly) in the UK, so my walls flow left to right. I select the parts of the project that help to string the process together, so a newcomer can see the highlights and get up to speed, without having to invest themselves in the detail.

4. Tell them what problem you are working on.

In a sentence, where possible. You should know this sentence!

5. Tell them what you’d like to see as outcomes.

If it’s a bit of internet, a prototype or tool, some training, a change in language in an organisation, a better way to exchange and use data … let people know the general area that this work might head in.

6. Tell them how you’ll be working.

A brief project plan is a nice way to share this. No gantt charts please! Just a run down of the steps you’d like to take (e.g time researching with users, time to make sense of the research and share, time developing ideas with users, rounds of feedback, show and tell frequency)

7. Show your highlights

Whether it’s important quotes from research, photos you’ve gathered of people’s homes, journey maps that help to point out the bits that are difficult and the places people want to thrive. Put these on your wall and label them. People want to know what a section of work is, and more importantly, why you did it, what did you learn from it.

8. Ask for feedback

If there’s a bit of the work that you’d like input in, ask for it. Make your questions clear, tell people why you are asking and make it easy for them to respond to you.

These are the walls I aim to create for my projects. I like making it part of my role to share the work with wider teams and interested parties, it challenges me to be succinct and approachable.
Debug service under high load
By: Alexander Koval
Tags: Best Practices, Debug, API, Troubleshooting, Service Design

The problem

Everyone understands the importance of logs. Anytime something weird going on logs became a starting point of investigation. And the better your logs are, the sooner you’ll identify the problem. However logs have certain limitations. Imagine you have operation with several stages and you’d like to see intermediate result of each stage. You might think that it would be a good idea to include stage outputs into logs. But what if this operation happens 3000 times per second? Your logs would soon became flooded with gigabytes of similar records and that will make them useless. You have to find another way to check on your system.

Aggregation is your friend

When it comes to big data we have to use aggregation in order to comprehend it. If I’m checking how repetitive operation is performing I don’t have to look at each individual iteration — I’d like to see the whole picture — how many operations been successful and how many failures happened. We might periodically write summary into logs but there is a better way — let the engineer to request this information explicitly rather that waiting until it pops up in logs. For this reason you can supply your system with service API that will provide certain reports on demand.

How to build useful reports

Identify your main metric of performance and the best representation for it

Choose the right units for your metrics

Think which asynchronous operations might became potential bottlenecks and collect time measurements for each of those

Make classification of errors and count error occurrences for each class

Think what kind of state changes you’d like to control during debug

Whenever you realize during debug session that you need extra-metric that is not present in service API — add it right away

Example

Let’s analyze the what Service API might look like and what information it provides us

Diagnostic API response

Above you can see an example of diagnostic response from high-load service I ever build. This is convertor service, its purpose was to retrieve instances from message queue, process them and store. Form this output I can see that 100 entities being fetched from queue, 95 successfully processed, 5 processing operations caused an error — out of those 3 errors of typeA and 2 errors of typeB. The bottleneck is operationD — takes 98.88% of processing time. Overall throughput is 3257 operations per second.

Now imagine how long would it take to identify missing messages and found bottleneck without having this report. Efforts to build service API will save you many hours during troubleshooting.

It is also helpful to include some state-changing commands into service API — sometimes I might want to order instance to stop fetching messages from queue. I used it to fill up queue slowly and then turn on consumer and test the maximal throughput.
Design for Public Health
By: Sayanee Halder
Tags: Public Health, Service Design, Conceptual, Generic Medicines, Digital Transformation

Making quality medicines available at affordable prices for all.

Healthcare and medicines are expensive for an average Indian household. More often than not, patients feel they are compelled to buy expensive branded medicines although there are generic medicines available in the marketplace. Not so long ago, a bunch of my colleagues and I participated in the socially conscious designathon held at the Deloitte Digital Studios, Mumbai. Our challenge — critically analyse the healthcare inequality in the country.

“When I visit a doctor, I do not get generic medicines prescribed and end up paying huge cost for specific medicines.”

What are generic medicines?

Generic medicines are bio-equivalents of their counterpart branded drug — essentially non-branded drugs with the same chemical formula/active ingredients. These medicines cost approximately 30–40% lesser and have the same effectiveness as branded drugs.

The Indian government has been running a program, Jan Aushadhi Pariyojna aimed at providing awareness and facilitating availability of generic medicines since 2008. While the campaign has had some successes it has had its fair share of snafus — procurement and supply chain logistics to be specific.

That got us thinking …

How can the choice of medicines, especially generics, be made available to patients when they seek treatment so that they are in control of the medicine cost? How can everyone in the ecosystem benefit from it?

The current public health ecosystem consists of government bodies and pharmaceutical companies on one hand; and doctors, patients, their friends/family and the local chemists on the other hand. The pillars of this ecosystem work and interact with each other uniquely. As rightly noted by Daniel Burka having worked in this domain extensively — the system needs to work in the individual level as well as in the group level. While the public health decision makers are concerned with the larger picture/numbers, a healthcare worker, a doctor or a chemist has a more direct, one-on-one relation with the seekers.
Basics of Digital Marketing Services:
By: Wecode Future
Tags: Digital Marketing Agency, Services, Digital Marketing Service, Digital Marketing, Service Design

Gone are the days when business owners still welcome the notion that Digital Marketing is only for the likes of multinationals and large corporations with the sufficient resources required to mount an online marketing campaign. Instead, Digital Marketing levels the playing field, providing small and medium enterprises the chance to compete against the big boys and attract their share of targeted traffic.

With digital marketing Services, small companies now have the resources to perform sales and marketing processes previously available only to large corporations. Without a call center, small businesses can engage effectively with multiple customers, even customers from any part of the world, even if they don’t have physical stores or branches in these locations.

How Can Digital Marketing Services Help Your Business Growth?

Digital Marketing levels up the online marketing game

It is more cost-effective than the traditional marketing

It helps in delivering conversions

It helps in generating business revenue

It facilitates interaction with the targeted audiences

It caters to the mobile consumers

It helps in building brand reputation

Provides better Rate Of Investment

It earns people’s trust

Furthermore, It entices people to take favorable action

It makes you ready for the internet of things

Ensures Online Business survival

Digital Marketing helps you use proven strategies and techniques that attract not necessarily more traffic but targeted traffic that delivers results.

Targeting the right kind of people who deliver the right results is what Digital marketing is all about! Ensuring Survival For Your Business.

Every day, brick-and-mortar business establishments encounter visitors entering their store, skimming through and inspecting products, and then leaving without buying anything. Many kinds of people like these come and go, but a targeted few will actually make a purchase and, if satisfied, will come back for more on a later date. If you get less of the latter and more of the few, sooner or later, your brick-and-mortar business will cease to exist.

Conclusion:

Your business can be likened to this. Even if you have tons of website visitors, but none of them ever convert, your business will also cease to exist. Digital Marketing helps you use proven strategies and techniques that attract not necessarily more traffic — but highly targeted traffic that delivers results. Targeting the right kind of people that delivers the right kind of results is what Digital Marketing is all about — ensuring survival for your business.

Digital Marketing leverages social media signals, social proof, and testimonials from actual consumers who have previously purchased, joined or availed of a product or service marketed by a particular brand or business. The more reliable these social signals are, the higher the trust rate it can generate from targeted audiences — most of which can be potential customers.
About Me
By: Peter Waters
Tags: Systemic Design, User Research, Intern, Service Design

Presenting a concept at London GovJam 2019

To get in touch, drop me an email at peter77waters@gmail.com or send me a message on LinkedIn.

I’m Peter, a designer and researcher in human system transitions. I draw from human-centred design, systems thinking and development theory in order to support organisations and communities to tackle complex socio-environmental problems.

Motivation

I excel when working on challenges that have a murky definition and where there is the power to change the direction of strategy or policy.

Some of the challenges that I have worked on include:

social isolation and division in urban areas

disaster resilience in diverse communities

loneliness in remote workers

elderly care in multi-location families

I enjoy working with multiple stakeholders and prototyping interventions that have a real-world social component

policy design or project/programme design,

I welcome any opportunities to collaborate with consultancies with a social focus as well as organisations from the public sector and third sector.

Approach

My design approach is influenced by concepts such as systems thinking, co-production, feminism, environmentalism and asset-based community development.

My aim is for stakeholder to feel more empowered than they did before.

What sort of design

What systemic design means to me

Key skills

📐 Project design, development and evaluation

🦺 Behavioural intervention design and lean prototyping

🔬 Qualitative and quantitative research design

🙋‍♂️ Presenting and conducting workshops

🤝 Stakeholder management

🕵️‍♂️ Data analysis and insight synthesis

📝 Proposal and report writing

Experience

a human systems designer at the British Red Cross where I am working on a Nesta-funded project to improve community resilience.

I previously worked at IBM iX as a user experience design intern and at INDEED Innovation as a product design intern.

I am currently on a placement year for my BA Industrial Design degree at Loughborough University. If you have any openings for a service design internship before October 2020 or are interested in my work, please get in touch at peter77waters@gmail.com

Background

I am in the final year of a Bachelor of Arts in Industrial Design and Technology at Loughborough University.

I am due to study a Master of Science in Global Prosperity at UCL in September 2021
Let me know what to say to the emergency number
By: Daniele Catalanotto
Tags: Customer Service, Customer Experience, Emergency, UX, Service Design

We were in holidays in the Swiss alps. I was wandering around in the reception of our hotel while I waited for my wife to come down. So I watched the board where all touristic and key information are displayed. Yes, I was a bit bored. But then I saw something interesting.

A little poster said:

In case of emergency call the number 144. You are at the following adress…

The original poster written in French. From the Hotel Aux Milles Etoiles in Les Marécottes, Switzerland

The poster also showed where you can find the defibrilator and made it clear with a few photos and arrows.

Why this is smart service design

I’m on holidays, if something bad happens I’m not in my usual environnement. If I’m a foreigner, I might not know what the emergency phone number is in Switzerland, so that already is pretty smart.

But the thing that I found particularly smart is to mention the key information I should give to the emergency service. In this case, the adress.

This is something that you can also do when you show an error message that lets people know they should contact the support. You can add a little note that says:
Let’s be AMIGO’s
By: Ignacio Barboza
Tags: User Experience Design, User Experience, Mental Health, Design, Service Design

THE STORY

We’ve all felt frustrated, depressed and sometimes like there is no way out. In a developing region with little to no mental health awareness, Amigo Service Design becomes the first Latin American mental health platform to find the right therapist and counselling, match based on Availability, Geolocation and Needs.

“ People living in poverty have more limited access to adequate mental health care, thereby worsening the vicious cycle of inequality in Latin American countries. The first problem is the stigma surrounding mental illnesses. This makes it much more difficult for people to seek help because they feel ashamed or are concerned about the treatments. ”

“ Depression is the most common mental illness in the world. In Latin America, 5% of the adult population suffers from it. ”

— The World Bank

THE USER

To start my project I would like to share the point of entry of my service through a teaser video. Let’s imagine you were a young person in Latin America who encounters this type of promotional material through Instagram…

THE JOURNEY

USER JOURNEY

Through the practice of cognitive and social psychology Amigo platform becomes the first Latin American platform for the promotion of mental health. Design thinking is problem solving, this means that design can improve the quality of life.
Reflecting on Reflection
By: Xiaoran Sun
Tags: User Experience, Reflections, Service Design

Reflection, my new found skill which I had frequently subject myself to, learning on the fly despite my dispassion. Thinking about thinking. Common questions that crop up; Is this good? What does good look like? What do other’s think? Is my image the same in their mirror? If I am being honest, I hadn’t ever seen the benefits of reflection. Why waste time on the past when there is the future? Perhaps it is my upbringing, perhaps it’s my culture combined with an environment that has made me this way. This is how I felt until I came to the UK. In the beginning, it was a real challenge to fully understand and empathize with what tutors were trying to convey to me. Through practice and due diligence, I redesigned my own thought process to include and allow reflection to take place. I have found it to be enlightening.

We were given a personal opinion and aspirations exercise where each of us would highlight their strengths and weaknesses as well as hopes and desires within the professional world. Once we completed our own slide, we had to inspect what others presented and leave feedback comments or acknowledgements. This was to aid us in choosing a topic for the ‘finale’. I found it difficult thinking about myself so much, an image the Droste effect flashed with all the different possible versions of myself as seen by other people — looping forever as I reflected upon my reflection. I got over it and typed out who I thought I was.

The whole exercise was a comforting eye opener. My classmates agreed with me on my strong points and gave me helpful insights in regards to my weaknesses. It was interesting examining each person’s card, we were all equally exposed and this made the experience all the more important. I felt strong empathy for everyone and felt a lot closer to the entire MA Service Design roster. From that day, I feel that reflection is definitely a powerful and useful tool. One may never ‘master’ reflection, but I feel if you are truly honest when you look at yourself, then the image will smile back at you.
My take-aways from The Interaction Design Foundations Integrated Service Design course
By: Chanel Chomse
Tags: Design, Idf, Services, Learning, Service Design

Recently, I’ve taken up a few courses that the Interaction Design Foundation offer. As I work through them, I will be reflecting on my learnings. Sharing what I’m learning is my way of committing to doing so. Frank Spillers did an incredible job putting the course work together and I’d definitely recommend their courses to anyone from beginners to those looking to brush up on their skills.

Reflecting on what service design is

Reflecting on what service design is and what Service Designers do was a valuable exercise and served as a good reminder.

Defining what Service Design is and isn’t is a tricky thing and often because it seems to overlap with so many other design disciplines.

What I took away from it was:

As service is only as good as its failure points and a Service designers responsibility is to ensure that those are identified and considered with a fair degree of accuracy (that last bit is still up in the air for me).

Service Designers are there to ensure that a service is feasible to produce, scalable in size and financially viable.

Communicating what the service is, what it looks like and all of the touchpoints that need to align in order to make it work smoothly.

An integral part of a Service Designer’s role is putting all those pieces together in a way that makes sense for everyone involved. Seeing the problem, identifying the opportunities, testing ideas and laying the foundation for the service to be built and tested further.

so, is it a product or is it a service?

The interaction design foundation speaks of the value customers place on a product when it is purchased; the exchange of money for ownership or possession of a thing. When customers buy a service, they’re exchanging money for a ‘performance’. The ownership of the ‘performance’ is not exchanged.

I loved the example of Frank’s example of purchasing a cup of coffee — it really got me thinking about the services I engage with.

Buying a cup of coffee requires the exchange of money for possession over a cup of coffee but we might choose the coffee shop we go to based on a variety of other factors but for now, we’ll focus on service delivery. That is the ‘performance’ one places additional value on.

It opened up a series of questions — let’s think about something like Netflix for example. Is Netflix a product or a service? Technically, I’d argue that Netflix is a digital product but when I think about it, it’s evolved into a product as a service. You exchange money every month but you don’t actually own the content and the commitment is minimal. There are some digital theatrics and algorithms that deliver a good service experience.

Food for thought — Think about something you’ve bought recently and ask yourself: is it a product, a service or both?
ARIMA simplified.
By: Abhishek Rajbhoj
Tags: Machine Learning, Arima, Stock Market, Time Series Analysis, Data Science

ARIMA simplified.

Background

The fundamental intuition behind time series forecasting is that, the measure of some variable at a time period will depend on the measure of the same variable at a previous time period, 2 time periods prior, 3 … and so on.

Time series forecasting finds its applications in a lot of prediction problems (especially in the business sector) and one of the best places to start is ARIMA!

The goal of this article is to highlight the building blocks (i.e. AR, I and MA) and the underlying concepts (i.e. auto-correlation and partial auto-correlation) behind this model.

Correlation and Lag Factor

Let’s consider an example of predicting the price of gas at any particular day, say Sunday. It’s obvious and easy to think of, that Sunday’s price will be dependent on Saturday’s price.

Now let’s consider the effect of Friday’s price on Sunday’s price

As Sunday’s price is dependent on Saturday’s (2), which in-turn is dependent on Friday’s (1), there could be an indirect transitive relation between price on Friday and price on Sunday.

Aforementioned relations are visualised in the diagram below.

Auto-correlation

Apart from these, there exists one more relation which might be difficult to imagine at first i.e. the direct effect of Friday’s price on Sunday’s price (3).

This mutual relationship between day prices here, is called correlation. If values of two variables increase (or decrease) together then they are said to have a positive correlation. If the value of one variable increases and the other decreases (or vice versa) then they have a negative correlation.

Correlation is an important concept of time series and it has 2 flavours,

Auto-correlation, which considers both direct and indirect effects (illustrated in the diagram above) and,

which considers both direct and indirect effects (illustrated in the diagram above) and, Partial Auto-correlation, which considers only the direct effects like in the figure below
What Is Time Series, And Why Is It Special?
By: Thomas Oriol
Tags: Machine Learning, Forecasting, Timeseries, Time Series Analysis

Our definition of times series is: sequentially revealed, time-stamped and time-critical data.

Examples include stock prices, logistics scans, machine logs, retail sales, website visits… Counter-examples include images, speech, consumer preferences, social graphs…

Why do machine learning algorithms applied to time series underperform when using standard training, validation and testing procedures?

This comes from three specificities of times series.

They have no stable underlying structure. In other (technical) terms, you can’t assume that data points are independent and identically distributed (“IID”). It is safer, and more realistic, to assume frequently shifting time dependencies. Their sequence matters for learning. Suppose you are learning to recognize cats based on one million pictures; the order in which your algorithm processes them won’t change the outcome of the learning process. But if you take a stock chart, break it into daily variations and feed these in random order to your algorithm, there will be nothing useful left to learn. Their sequence matters for interpretation. With time series, the sequence of events is as meaningful as the events themselves. For example, the fact that a delivery delay occurred today and not yesterday is potentially crucial information for a supply chain manager. Whereas the fact that I posted my picture today and not yesterday doesn’t affect Facebook’s ability to tag me.

The standard training, validation and testing procedure is based on the assumption that data points are IID. And standard feature engineering techniques don’t work natively on sequential data…

Hence the two usual consequences of ignoring the specificities of time series when building your machine learning solution:
LSTM Framework For Univariate Time-Series Prediction
By: Joseph, Iosif
Tags: Machine Learning, Deep Learning, Time Series Analysis, Python, Keras

Code & Data Walkthrough — Data Prep

The data is the US/EU exchange rate from 2010 to present, not seasonally adjusted. You can pull the data from the St. Louis Federal Reserve here.

The Data

Before building the model, we create a series and check for stationarity. While stationarity is not an explicit assumption of LSTM, it does help immensely in controlling error. A non-stationary series will introduce more error in predictions and force errors to compound faster.

We filter out one ‘sequence length’ of data points for later validation. In this case, 60 points.

Our Dickey-Fuller — Shows Stationarity

The data format required for a LSTM is 3 dimensional, with a moving window.

So the first data point will be the first 60 days of data.

The second data point is the first 61 days of data but not including the first.

The third data point is the first 62 days of data but not including the first and second.

Our Training Data Frame

The last major step of prep is to scale the data. Here we use a simple minmax scaler. Our sequence length is 60 days for this part of the code.

Code Walkthrough — Building the LSTM

The next part of the analysis is pretty straightforward code-wise. We first create a learning rate scheduler. This scheduler will monitor the validation loss and modify the learning rate on plateaus.

We create a method to reset all of the weights in case we want to re-train with different parameters (the method is unused in my code but it’s there if you need it).

We then build an LSTM with 2 layers, each with 100 nodes and build an output function with a sigmoid activation.

The history object stores model loss over epoch, which can be plotted to evaluate whether an adjustment is needed in the training process.

Model Loss by Epoch

Code snippet
How Holt’s Winter Beats out ARIMA when there is limited Data
By: Narendranath Nadig
Tags: Machine Learning, Data Science, Time Series Forecasting, Time Series Analysis

Hi guys, as most of you know ARIMA and Holt’s winter are two popular Time series prediction models which are used to predict stationary/non-stationary data.

A project I was working on Involved comparing and predicting the water consumption for 84 areas in my city.I used 6 time series models, but I am limiting my discussion for only the two popular seasonal data prediction models.

These are the graphs I got for a particular area

Water Consumption prediction using Holt’s Winter Method

Water Consumption prediction using ARIMA

As you can clearly see, the amount of data I had was very limited,25 months to be precise.Despite a lack of data,The Holt’s winter could somehow catch the shape of the consumption prediction, where as ARIMA turned out to be a straight line with a negative correlation, something which we wouldn’t want .

Another thing which was notable was the fact that the RMSE value of Holt’s winter was less than the ARIMA predictions in 73 of the 84 DMAs I measured.

To conclude,I would like to tell all time series Analysts to always use Holt’s Winter over ARIMA when there is a lack of data.
Efficient Time-Series Analysis Using Python’s Pmdarima Library
By: Muriel Kosaka
Tags: Machine Learning, Time Series Analysis, Python, Data Science, Time Series Forecasting

Efficient Time-Series Analysis Using Python’s Pmdarima Library

Photo by Aleksei Zaitcev on Unsplash

What is Time-Series Analysis?

One of the key concepts in data science is time-series analysis which involves the process of using a statistical model to predict future values of a time series (i.e. financial prices, weather, COVID-19 positive cases/deaths) based on past results. Some components that might be seen in a time-series analysis are:

Trend : Shows a general direction of time series data over a period of time — trends can be increasing (upward), decreasing (downward), or horizontal (stationary). Seasonality : This component exhibits a trend that repeats with respect to timing, magnitude, and direction — such as the increase in ice cream sales during the summer months or increase in subway riders during colder months. Cyclical Component : A trend that has no set repetition over a certain time period. A cycle can be a period of ups and downs, mostly seen in business cycles — cycles do not exhibit a seasonality trend. Irregular Variation : Fluctuations in time-series data that is erratic, unpredictable and may/may not be random.

When conducting time-series analysis, there are either Univariate Time-Series analysis or Multivariate Time-Series analysis. Univariate is utilized when only one variable is being observed against time, whereas Multivariate is utilized if there are two or more variables being observed against time.

What is ARIMA? Why Use Pmdarima?

ARIMA is an acronym which stands for Auto Regressive Integrated Moving Average and is a way of modeling time-series data for forecasting and is specified by three order parameters (p,d,q):

AR(p): pattern of growth/decline in the data is accounted for

I (d): rate of change of the growth/decline is accounted for

MA (q): noise between time points is accounted for

There are three types of ARIMA models, ARIMA, SARIMA, and SARIMAX which differ depending on seasonality and/or use of exogenous variables.

Pmdarima‘s auto_arima function is extremely useful when building an ARIMA model as it helps us identify the most optimal p,d,q parameters and return a fitted ARIMA model.

As a newcomer to data science, when conducting time-series analysis, I took the “long” way before coming across pmdarima’s auto_arima function to build a high performance time-series model. For this article, I will focus on the Univariate Time-Series analysis to forecast the number of airline passengers (from Kaggle) and discuss through the traditional ARIMA implementation versus the more efficient, auto_arima way.
4 Must-Know Python Pandas Functions for Time Series Analysis
By: Soner Yıldırım
Tags: Machine Learning, Programming, Time Series Analysis, Python, Data Science

4 Must-Know Python Pandas Functions for Time Series Analysis

Photo by Luke Chesser on Unsplash

Time series data consists of data points attached to sequential time stamps. Daily sales, hourly temperature values, and second-level measurements in a chemical process are some examples of time series data.

Time series data has different characteristics than ordinary tabular data. Thus, time series analysis has its own dynamics and can be considered as a separate field. There are books over 500 pages to cover time series analysis concepts and techniques in depth.

Pandas was created by Wes Mckinney to provide an efficient and flexible tool to work with financial data which is kind of a time series. In this article, we will go over 4 Pandas functions that can be used for time series analysis.

We need data for the examples. Let’s start with creating our own time series data.

import numpy as np

import pandas as pd df = pd.DataFrame({

"date": pd.date_range(start="2020-05-01", periods=100, freq="D"),

"temperature": np.random.randint(18, 30, size=100) +

np.random.random(100).round(1)

}) df.head()

(image by author)

We have created a data frame that contains temperature measurements during a period of 100 days. The date_range function of Pandas can be used for generating a date range with customized frequency. The temperature values are generated randomly using Numpy functions.

We can now start on the functions.
Forecasting Stocks and Crypto prices using Redis, Prophet, and Grafana
By: Mikhail Volkov
Tags: Grafana, Redis, Prophet, Time Series Analysis, Forecast Market

Build Financial Application on RedisTimeSeries

In the Redis Labs blog post Build Your Financial Application on RedisTimeSeries, Prasanna Rajagopal explained how to use RedisTimeSeries to store, aggregate, query stock prices, financial information, and technical indicators — and many other time-series data sets used by investors.

RedisTimeSeries is schemaless, which means that you can load data without defining schema, add new fields on the fly, or change your data model should your business circumstances change. Its real-time performance and simple developer experience makes it fun to work with time-series data!

I would highly recommend reading it to learn more about RedisTimeSeries.

Facebook Prophet Model

The prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.

There are a lot of exciting articles describing how to use Prophet for forecasting and visualizing time series using standalone Python applications to get started.

Using RedisGears to forecast RedisTimeSeries data using Prophet model

Redis Labs announced RedisGears 1.0 in May last year, and one of the examples was the ProphetGears project, visualizing RedisTimeSeries with RedisInsight, which was the only option at that time.

ProphetGears project visualizing Forecast using RedisInsight.

I already wrote about adopting RedisGears for other projects but have not had a chance to try it with Prophet until now.

Prophet team recently (March 28, 2021) released Version 1.0 with many improvements, and I am excited to play with it.

Redis-Prophet Docker image

To start playing with Prophet, it should be installed using RedisGears and registered in the Redis database. It’s fast and easy to do on the Production server with high RAM, a powerful CPU, and Redis Enterprise. For development with limited resources, I prefer to use Docker containers that are easy to manage and recreate.

We developed a Docker image based on the latest version of RedisTimeSeries and RedisGears with Prophet pre-installed. This Docker image is automatically built every night using GitHub Action and can be used for any project.

To start Redis container using Redis-Prophet image run:

$ docker run -p 6379:6379 --name=redis-prophet ghcr.io/redisgrafana/redis-prophet:latest

RedisGears with Prophet may be warmed up to load dependencies and save the state in the database before running prediction in the redis-cli tool:

RG.PYEXECUTE "GearsBuilder().run()" REQUIREMENTS Prophet

Or in the Grafana with Redis Application plugin installed:

RedisGears Script Editor for Grafana.

Execute RG.PYDUMPREQS command to validate that Prophet and other dependencies loaded correctly:
What if I tell you RDBMS can handle time-series data?
By: Jyoti Dhiman
Tags: Data, Time Series Analysis, Timescaledb, Database, Design

Given high ingestion rates and near-real-time aggregates, relational databases cannot deal with the above-mentioned data considerations directly. But with some padding to the relational databases we can make it very powerful so as to handle time-series data. The relational database we’ll talk about now is postgresql and the padding that is referred to is its extensions.

We’ll talk about two such extensions below.

TimeScale DB

TimeScale DB is an open-source extension to PostgreSQL. Since it is bundled as an extension to postgresql, it can support all things postgresql like relational queries, full SQL support(not SQL-like) as well as the support of real-time queries.

How timescaledb handles time series data

It has a concept of hypertables which are simple chunks of data by time and stored in standard tables. These hypertables are abstraction layers for users and one can simply query, join or do any other operations on this table as done on a normal table.

Ingestion to these hyper-tables is very fast. As per documentation, it supports an ingestion of 1.5M+ metrics per second per server.

Also, for continuous aggregations materialized views can be created over these hyper tables such that aggregations are available near real-time. Internally, timescale DB has background processes that take care of refreshing the view and this refresh interval and parameters are configurable. It also provides integration with Kafka, kinesis, etc for data ingestion.

A materialized view is a database object that contains the results of a query

How to create real-time visualizations over this time series data?

The materialized views which help in providing real-time aggregates can be used to create real-time visualization too. Since the extension is bundled with PostgreSQL, it can be integrated with any visualization tool such as Graphana and to refresh continuous aggregates.

So, now with this simple architecture, you know how to ingest time series data and even create visualizations over it so it can be demonstrated such as in those big monitors in the office(well, if we ever go to the office, fingers crossed). Pretty simple, isn’t it? With just a relational layer?

In addition to that, it also has methods for forecasting time-series data too! The documentation of timescaledb is very rich and can be followed along to do a proof of concept.

Pipeline DB

Pipeline DB is another extension that PostgreSQL supports, which also can be used for real-time reporting, and similar to timescaledb supports real-time aggregations in materialized views over time series data.

The things to note about pipeline DB are its efficiency comes from it not storing raw data, so if you need raw data to be stored(IMO, it should always be stored). But it looked promising too and serves the use case so worth mentioning here.
ML Approaches for Time Series
By: Pablo Ruiz
Tags: Machine Learning, Ml Time Series, Time Series Analysis, Machine Learning Approaches for Time Series Data, Time Series Forecasting

The picture shows only the axes of time and the response. Remember that, in our case, there is 3 more variables which are the responsible for the values of the t.

The picture in the top shows a windows of a chosen (and fixed) size w, which in this case is 4. This means that, the model is going to map the information contained in that windows, with the prediction at point which is at t+1. There is an r in the size of the response, because we could want to predict several time steps in the past. This would be a many-to-many relationship. For simplicity and easier visualization, we will work with r=1 .

We can see now the effect of Sliding Window. The next pair of inputs-outputs that the model would have for finding the mapping function is obtained by moving the window one time step to the future, and proceed the same as we did at the previous step.

Ok then. How do we apply this to out current dataset? Let’s look at what we need, and build our helper functions.

But first, we don’t want time to be absolutes values, we are more interesting in knowing which is the elapsed time between observations (remember the data is NOT evenly spaced!). Therefore, let’s create a ∆t and take a look at our data.

dataset = pd.DataFrame(np.concatenate((t, x1, x2, x3, y), axis=1),

columns=['t', 'x1', 'x2', 'x3', 'y'])



deltaT = np.array([(dataset.t[i + 1] - dataset.t[i]) for i in range(len(dataset)-1)])

deltaT = np.concatenate((np.array([0]), deltaT))



dataset.insert(1, '∆t', deltaT)

dataset.head(3)

Now that we know how our dataset will look like that, let’s recreate what we want our helper function to do over a scheme of a table.

For a window of size 4:

What our function will do is to flatten all the information contained in the window, which is all the values inside the W window, and the timestamp(s) of when we want to make the prediction.

This way, we could have 2 different equation to model our system, depending if we include previous values of the responses as new predictors.

The result that the function have to return should look like:

We will be able to create l = n - (w+r) +1 windows, because we lose the first rows, as we don't have previous information for the first value of Y(0).

All the lags we have mentioned acts like new predictors for the model (in this visualization the previous values of Y are not included, they will follow the same as the Xi. Then, the timestamps (elapsed) where we want the prediction to be done ∆t(4), and the corresponding value of what the prediction should be Y(4). Note that all the first ∆t(0) are initialized to 0 as we want to standardize every window to be in the same ranges.

Here is the code created to achieve this process. There is a function WindowSlider from which we can create objects to construct different windows changing the parameters.