teleren/devto huggingface dataset:
Days 1 and 2 - Auth
Tags: startup, oauth, refactor, security
Published: 2020-05-28 03:19:54

It wasn't so bad the past couple of days. A stable core product means I can stay head down in some code. And I definitely needed the focus.

Being a startup, we've moved fast and cut a couple corners here and there over the years. Nothing critical, just not the cleanest code in some places... and of course, that will eventually come back to bite you in the ass.

After the past couple of days, my ass looks like hamburger meat. I've been bitten a lot. Some key findings:

- hardcoded dependencies on Slack being the OAuth provider
- assumptions that we'd have Slack or a similar workplace messaging tool as the initial authentication for new users (with a "team"-like structure)
- duplicate code copy/pasted in multiple places

Obviously some of the fixes were easy, like the copy/pasted code. Just a little refactoring there. The hardcoded dependencies and assumptions were a little trickier, and required some changes up and down the stack from front-end to database.

I'm happy to say we've got the Google OAuth implemented and working, intertwined with all the setup steps we take for new Slack users.

![Sign in with Google](https://dev-to-uploads.s3.amazonaws.com/i/de1us4285y2bx719zh9c.png)

With the groundwork now set for multiple auth providers, I feel better when inevitably we need to add another "Sign in with XYZ Service".

Random pro-tip: if you're manually setting cookies to store something temporary, make sure you mark `HttpOnly` and `Secure` to `true`.
Get top articles from dev.to with 🎩 Alfred
Tags: python, alfred, macos, devto
Published: 2020-05-28 03:06:15

I use [Alfred](https://www.alfredapp.com/) with many workflows every day, is a very good tool for boost your productivity if you make the initial effort to be used to the change.

Another thing that I like to do is to read the top articles on [dev.to](https://dev.to) but I didn't found any Alfred workflow for dev.to, so this was a great opportunity to build something useful (at last for me) and as part of my [#100DaysOfCode](https://mijailr.com/days_of_code/) challenge I decided to create a workflow for dev.to.

There is an Open API to get some information from dev.to, so using a simple query you can get the top 10 articles for the day:

```bash
curl https://dev.to/api/articles?top=1&per_page=10 | jq
```

My first thought was to use a bash script to get the articles and parse everything with `jq` but since `jq` doesn't come pre-installed on MacOS, so searching on the web about the best way to create an Alfred Workflow, wasn't hard to find an excellent library called [alfred-workflow](https://github.com/deanishe/alfred-workflow), created an maintained by Dean Jackson (@deanishe), so cheers to him because he made the harder part.

This library was written on Python so the answer to the question _How we can make a high quality Alfred workflow?_ is with Python and `alfred-workflow`. Also the documentation is very good so is easy to start.

Please review the code here [github.com/mijailr/alfred-devto](https://github.com/mijailr/alfred-devto).

Or if you prefer, you [📦 download the workflow here](https://github.com/mijailr/alfred-devto/releases/latest/download/devto.alfredworkflow)
Generate beautiful blob shapes for web and flutter apps
Tags: flutter, javascript, showdev, design
Published: 2020-05-28 03:02:44

---
title: Generate beautiful blob shapes for web and flutter apps
published: true
description: 
tags: flutter, javascript, showdev, design
cover_image: https://user-images.githubusercontent.com/1754676/83091868-cef86280-a0b9-11ea-9b29-5b60c6df44ce.png
---


Recently I was trying to convert a dribbble shot to a flutter app, and then quickly realised blob shapes in the design is not something easy to bring it in flutter. Initially I started with SVG's from [Blobmaker](https://www.blobmaker.app/) but animation and morphing the path became a pain, because I am beginner and I truly hate anything that involves Math.

So, I decided to create a package that fits to my needs and some common use cases.

### Blobs for flutter
[flutter_blobs](https://github.com/lokesh-coder/flutter_blobs) is a tiny package which helps to create and animate blobs shapes. There are quite a few features in it, like clipping, children widgets, gradient background, outlines and couple more. There is no dependencies and it is flexible, as the path can be manipulated easily to create complex shapes.

<table><tr><td><img src="https://user-images.githubusercontent.com/1754676/82777008-6ebfb180-9e6a-11ea-9b85-fcac09cfca38.gif"/></td><td><img src="https://user-images.githubusercontent.com/1754676/82777157-ef7ead80-9e6a-11ea-9b74-6224cf33d739.png" width="200"/></td>
<td><img src="https://user-images.githubusercontent.com/1754676/82777018-75e6bf80-9e6a-11ea-913d-873e9ba4444c.gif" width="200"/></td>
</tr></table>

{% github lokesh-coder/flutter_blobs %}

### Blobs for Web
Random blob shapes are great in some scenarios, but in most cases user always wanted a same blob shape on every render. This is where things got complicated. For that I have to build a web app, where user can generate blob shapes and use it in flutter app with a ID. My first idea was to build a web app with flutter web. But things didn't worked well, as there were lot of challenges like slow initial load, less customisation. So, finally I ported the path generation logic to Javascript and made a simple Blob generation tool using the SVG.

{% twitter 1265664002102824968 %}

[https://blobs.app/](https://blobs.app/)

### Takeaway
As I mentioned earlier, Math is just a pain for me. But stepping out and learning trigonometry and pseudorandom number generator (PRNG) was a very good experience.

### Links
Web app - [Blob generator](https://blobs.app/)
Flutter package - [blobs](https://pub.dev/packages/blobs)
Package repo - [flutter_blobs](https://github.com/lokesh-coder/flutter_blobs)

### Contribution
I would really like any kind of contribution from the community.
Placeholder title
Tags: gftwhackathon
Published: 2020-05-28 02:52:26

[Instructions]: # (To submit to the Grant For The Web x DEV Hackathon, please fill out all sections.)


## What I built


### Submission Category: 

[Note]: # (Foundational Technology, Creative Catalyst, or Exciting Experiments)


## Demo
 

## Link to Code

[Note]: # (Our markdown editor supports pretty embeds. If you're sharing a GitHub repo, try this syntax: `{% github link_to_your_repo %}`)


## How I built it 

[Note]: # (For example, what's the stack? did you run into issues or discover something new along the way? etc!)


## Additional Resources/Info 


[Reminder]: # (We hope you consider expanding your submission into a full-on application for the Grant for the Web CFP, due on June 12.)
Machine Learning - Outliers - Dealing and Detecting it
Tags: machinelearning, datascience
Published: 2020-05-28 02:31:24

HOW TO DETECT OUTLIERS?

A. Detecting outliers are having many methods divided broadly into three types,

    1. Supervised methods
    2. Semi-supervised methods
    3. Unsupervised methods

B. Simply visualizing the data can also help us to find outlier. Visualization methods like Box-plot, Histogram and Scatter-plot can also be used.

C. Any value beyond the range of -1.5*IQR to 1.5*IQR (Inter-Quartile Range)

D. Using capping methods, any value which is out of range of 5th and 95th percentile can be considered as an outlier.

E. Data points, which are three or more standard deviations away from mean are considered to be an outliers.

F. Outlier detection is merely a special case of the examination of data for influential data points and it also depends on the business understanding.

DEALING WITH OUTLIERS:

I read an article on outliers and found it very interesting and I am sharing that in this post. There are four major ways of dealing with an outlier. They are as follows,

1. Deleting Observations: We delete outlier values if it is due to a data entry error, data processing error or outlier observations are very small in numbers. When there is no importance for these points in the model building can also be detected.

2. Transforming and binning values: Transforming variables can also eliminate outliers. Natural log of value reduces the variations caused by extreme values. Binning is also a form of variable transformation. Decision Tree algorithm allows dealing with outliers well due to binning of a variable. We can also use the process of assigning weights to different observations and you can also use a sigmoid function to squash values.

3. Imputing: We can also impute outliers. We can use the mean, median, mode imputation methods. Before imputing values, we should analyse if it is a natural outlier or artificial. If it is artificial, we can go with imputing values. We can also use a statistical model to predict values of outlier observation and after that we can impute it with the predicted values.

4. Treat Outliers separately: If there are significant number of outliers, we should treat them separately in the statistical model. One of the approaches is to treat both groups are two different groups and build an individual model for both groups and then combine the output.
What is the best frontend solution, considering SEO?
Tags: webdev, seo, discuss
Published: 2020-05-28 02:04:54

I think CMS's are moving away from SEO, targeting only backend simplification.

For frontend, plugins and community packages included, what is the best for SEO? Is it Gatsby? What about Hugo?

What about CMS integration? Does it have to be GraphQL? Or, is it strict to flat file?

We also have to consider on "how to move away from boilerplates" as well, in order to have only what you expect, and not making all websites look the same.

If SSG, I think will also have to consider CI/CD integration to hook into new deployment, and is incremental build possible?

Also, can SSG?

- Hook into API?
- JavaScript integration?

All in all, the bottom line is still how to move into the first Search Engine page...
Go - Error
Tags: go
Published: 2023-08-13 02:09:00

---
title: Go - Error
published: true
date: 2020-05-28 00:00:00 UTC
tags: go
canonical_url: https://manon.icu/blog/go/10-errors
---

Go 是一种编程语言，它提供了一种简单而有效的错误处理机制。 Go 的错误处理主要基于两个概念：错误类型和错误值。

在 Go 中，错误类型是一个接口类型，定义了一个名为 `error` 的预定义接口。该接口只有一个方法 `Error()`，它返回一个字符串，表示错误的描述信息。任何实现了 `error` 接口的类型都可以被视为一个错误类型。

错误值则是实现了 `error` 接口的具体实例。当出现错误时，通常会返回一个错误值来表示错误的发生。在函数调用中，通常会将错误作为函数的最后一个返回值返回。如果没有错误发生，错误值为 `nil`。

下面是一个简单的示例，展示了如何在 Go 中处理错误：

```go
package main

import (
	"errors"
	"fmt"
)

func divide(x, y int) (int, error) {
	if y == 0 {
		return 0, errors.New("division by zero")
	}
	return x / y, nil
}

func main() {
	result, err := divide(10, 2)
	if err != nil {
		fmt.Println("Error:", err)
	} else {
		fmt.Println("Result:", result)
	}
}
```

在上面的示例中，`divide` 函数用于执行整数除法运算。如果除数为零，则返回一个错误值，表示发生了除以零的错误。在 `main` 函数中，我们调用 `divide` 函数，并使用错误值进行错误检查。如果错误值不为 `nil`，则打印错误信息；否则，打印计算结果。

Go 还提供了一种更简便的错误处理方式，即使用 `errors` 包中的 `Errorf` 函数。该函数类似于 `fmt.Printf`，可以通过格式化字符串和参数创建一个新的错误值。例如：

```go
return nil, fmt.Errorf("error occurred: %v", someValue)
```

这样可以更方便地生成自定义的错误信息。

此外，Go 还支持自定义错误类型。开发人员可以通过定义满足 `error` 接口的自定义类型来表示特定的错误。这样可以提供更丰富的错误信息和错误处理逻辑。

Go 的错误处理机制基于错误类型和错误值。通过返回错误值，并使用条件语句进行错误检查，开发人员可以有效地处理和传播错误信息。

除了使用错误值进行错误检查外，Go 还提供了另外两种错误处理方式：错误恢复（Error Recovery）和错误日志记录（Error Logging）。

## 错误恢复

在某些情况下，当程序遇到错误时，我们可能希望程序能够继续执行而不是立即终止。为了实现这种错误恢复的能力，Go 提供了 recover 函数和 panic 机制。panic 用于引发一个错误，并立即停止当前函数的执行，然后程序会开始回溯执行调用栈，直到遇到包含 recover 函数的延迟函数调用。recover 用于捕获 panic 引发的错误，并允许程序进行一些处理。通过使用 defer 关键字和 recover 函数，我们可以在一些关键的代码段中设置恢复机制，以确保程序在遇到错误时能够继续执行，并进行一些必要的清理操作。

```go
func process() {
	defer func() {
		if err := recover(); err != nil {
			fmt.Println("Recovered from error:", err)
		}
	}()

	// 执行可能引发错误的代码
	// 当遇到错误时，使用 panic 引发错误
	panic("something went wrong")
}
```

process 函数中的 panic 语句会引发一个错误，并立即停止函数的执行。但由于设置了延迟函数调用和 recover 函数，程序会继续执行 defer 块中的代码，捕获错误并进行处理。

## 错误日志记录

除了简单地检查错误和恢复错误外，有时我们还需要记录错误信息以进行故障排查和日志记录。在 Go 中，通常使用标准库中的 log 包或第三方日志库（如 logrus、zap 等）来记录错误日志。这些库提供了各种级别的日志记录功能，可以将错误信息和其他相关信息写入日志文件或其他输出目标。

```go
package main

import (
	"errors"
	"log"
)

func main() {
	err := someFunction()
	if err != nil {
		log.Println("Error:", err)
	}
}

func someFunction() error {
	return errors.New("something went wrong")
}
```

使用标准库中的 log 包将错误信息记录到日志中。可以使用不同的日志级别（如 Println、Printf、Error 等）根据需要进行日志记录。

如果在错误恢复机制中没有设置延迟函数调用和 recover 函数，那么在 panic 引发错误后，程序将会立即终止并打印出错误信息。这种情况下，错误将无法被捕获和处理，程序会退出。

例如，考虑以下代码：

```go
func process() {
	// 执行可能引发错误的代码
	// 当遇到错误时，使用 panic 引发错误
	panic("something went wrong")
}

func main() {
	process()
	fmt.Println("After process")
}
```

在上述示例中，当 process 函数执行到 panic 语句时，程序会立即终止，并输出以下错误信息：

```shell
panic: something went wrong

goroutine 1 [running]:
main.process()
	/Users/user/main.go:5 +0x39
main.main()
	/Users/user/main.go:10 +0x20
exit status 2
```

可以看到，错误信息指示发生了 panic，并显示了调用栈的信息。然后，程序会退出，并不会继续执行 fmt.Println("After process") 这行代码。

因此，在错误恢复机制中，如果没有设置延迟函数调用和 recover 函数，panic 引发的错误将会导致程序的立即终止。为了能够恢复错误并继续执行程序，必须使用 defer 关键字和 recover 函数来捕获和处理错误。
Kafka: produtores idempotentes
Tags: kafka, kafkabr, java, apachekafka
Published: 2020-05-28 01:25:58

_"...[idempotência](https://pt.m.wikipedia.org/wiki/Idempot%C3%AAncia) é a propriedade que algumas operações têm de poderem ser aplicadas várias vezes sem que o valor do resultado se altere após a aplicação inicial."_

---

Através de uma configuração simples no produtor Kafka, pode-se alcançar a idempotência na produção de eventos, eliminando completamente a possibilidade de duplicatas nos tópicos.

---

Uma das grandes qualidades do Kafka é a possibilidade de configurar a confiabilidade no produtor, que permite ajustes finos para atender com maestria aos requisitos não-funcionais dos casos de uso.

Uma dessas possibilidades é o produtor idempotente. Com ele existe a garantia de que não haverá duplicatas em nível de partição.

Essa garantia entra em ação quando a seguinte propriedade é definida como `true`:

```properties
enable.idempotence=true
```

Ao habilitar a idempotência no produtor, também será mandatório definir as seguintes configurações:

```properties
acks=all
max.in.flight.requests.per.connection=5
```

Com este setup, cada requisição que envia lotes de eventos ao Kafka terá um identificador único.

> Cada lote na requisição tem como destino uma partição e um tópico

Ao chegar no kafka, esta requisição será processada e cada lote enviado ao líderes das respectivas partições. Ao terminar isso, o `ack` é devolvido ao produtor.

Mas esta é uma comunicação em rede. Então o atraso em receber o `ack` ou qualquer outro problema poderá provocar a retentativa padrão dos produtores Kafka. Assim, ele enviará novamente a requisição identificada.

O broker ao receber novamente àquela requisição processada com sucesso, apenas devolve o `ack`, sem persistir novamente e assim evitando a duplicação.

Sem o produtor idempotente, o broker receberia a requisição da retentiva como se fosse uma nova, persistindo novamente o lote de eventos, causando duplicatas.

---

É isso pessoal!

Até o próximo!!
Disabling Cognito User Pools Authentication for a Single Mutation with AppSync
Tags: apsync, aws, serverless, cognito
Published: 2021-04-01 21:54:26

---
title: Disabling Cognito User Pools Authentication for a Single Mutation with AppSync
published: true
date: 2020-05-28 00:00:00 UTC
tags: apsync, aws, serverless, cognito
canonical_url: https://www.goingserverless.com/blog/disabling-cognito-user-pools-authentication-for-a-single-mutation-with-appsync
---

Typically I combined AppSync with Cognito User Pools for authorization. This works great for API's where the user is logged in but what if you need a few queries or mutations to work for users who aren't logged in?

A common example of this is a mutation that allows people to register.

<!-- more -->

While AppSync doesn't allow unauthenticated requests you can use API key authorization to get around the need for a user to be logged in.

Start by setting up AppSync with Cognito User Pools as the default authorization mode and API key as an additional authorization provider. This will protect the API using Cognito User Pools authorization but allow us to enable API key authorization for some fields.

Next I need a schema with a mutation.

```graphql
type Account {
  id: ID!
  email: AWSEmail!
  name: String!
}

input RegisterAccountInput {
  email: AWSEmail!
  name: String!
  password: String!
  ## Other registration fields
}

type Query {
  findAccount(id: ID!): Account
}

type Mutation {
  registerAccount(input: RegisterAccountInput!): Account
}
```

By default the entire schema can only be accessed if the request uses Cognito User Pools authorization. By adding `@aws_api_key @aws_cognito_user_pools` to a type/field you can allow both authorization methods. If you only want to allow API key then you can use `@aws_api_key`.

In the following example I've limited access to the `registerAccount` mutation to only requests using API key authorization and you can only access the `id` field on the `Account` that it returns.

```graphql
type Account {
  id: ID! @aws_api_key @aws_cognito_user_pools
  email: AWSEmail!
  name: String!
}

input RegisterAccountInput {
  email: AWSEmail!
  name: String!
  password: String!
  ## Other registration fields
}

type Query {
  findAccount(id: ID!): Account
}

type Mutation {
  registerAccount(input: RegisterAccountInput!): Account @aws_api_key
}
```

For the user to access the other `Account` fields they would need to switch to Cognito User Pools authorization and use `findAccount`.
Stacks
Tags: algorithms, datastructures
Published: 2020-05-28 00:00:00

---
title: "Stacks"
date: "2020-05-28T00:00:00.000Z"
tags: ["algorithms", "datastructures"]
---

A [Stack](https://en.wikipedia.org/wiki/Stack_(abstract_data_type)) is  a linear data structure (i.e. its elements form a sequence) where operations are performed at the 'top' of the stack. The order may be <abbr title="First In, First Out">FIFO</abbr> (First In, First Out) or more commonly <abbr title="Last In, First Out">LIFO</abbr> (Last In, First Out).

An analogue in real life is a stack of plates -- with a FIFO approach, you take a plate from the bottom of the pile whereas with a LIFO approach you take a plate from the top. 

The stack data structure has a multitude of uses, an obvious one is your browser history (a <abbr>LIFO</abbr> stack). Typically, a stack implementation will provide at least three operations -- `push`, `pop`, and `peek`.

Pushing onto the stack can be visualised at follows:

![Stack push](https://dereklawless.ie/bb3663714721ac83ee9ba179aaf9c6ea/stack-push.svg)

Popping from the stack can be visualised at follows:

![Stack pop](https://dereklawless.ie/506479a6e0a36a8ec42d6760bcd4c224/stack-pop.svg)

## Complexity
🔔 Complexity is considered in terms of worst case.

### Time
|Insertion |Removal |Retrieval |Notes
|- |- |- |-
|Θ(1) |Θ(1) |Θ(1) |Retrieval Θ(1) if popping, otherwise Θ(n) to locate

### Space
| |Notes
|- |-
|Θ(n) |The elements in the stack

## A <abbr>LIFO</abbr> implementation

Here we're using an array as a backing store for simplicity (although you can of course use a different approach such as a [linked list](https://dereklawless.ie/linked-lists/)).

Define a `Stack<T>` class with the following methods:

- `isEmpty` - determines whether the stack has elements
- `length` - returns the number of elements on the stack
- `peek` - returns an element from the stack without popping it
- `pop` - pops from the stack
- `push` - pushes onto the stack

~~~typescript
// TypeScript
export class Stack<T> {
	private elements: T[];

	public constructor() {
		this.elements = new Array<T>();
	}

	public isEmpty(): boolean {
		return this.elements.length < 1;
	}

	public length(): number {
		return this.elements.length;
	}

	public peek(): T {
		return this.elements[this.elements.length - 1];
	}

	public pop(): T {
		if (this.elements.length > 0) {
			return this.elements.pop();
		}
	}

	public push(item: T): void {
		this.elements.push(item);
	}
}
~~~

[The above code is available on Github](https://github.com/dereklawless/algorithms-and-data-structures-katas/tree/master/src/data-structures/stacks)
Do Significant Work
Tags: career, beginners, productivity
Published: 2020-06-22 16:42:16

---
title: Do Significant Work
published: true
date: 2020-05-28 00:00:00 UTC
tags: career,beginners,productivity
canonical_url: https://tndl.me/blog/2020/significant-work/
cover_image: https://artvee.com/saconud/2020/05/10047pofm.jpg
---

The Stripe Press just published a beautiful reprint of Richard Hamming's [_The Art of Doing Science and Engineering_](https://press.stripe.com/#why-are-we-doing-this), which led me (and quite a few others, I imagine) to read some of his work for the first time. The written form of his lecture, "You and Your Research", is [chapter thirty of the book](https://press.stripe.com/#why-are-we-doing-this) (go read it if you haven't).

It opens with Hamming admitting that the lecture could just as easily be called “You and Your Engineering Career,” or even “You and Your Career". He suggests that we should try to do significant things in our careers, rather than insignificant things. Hamming means 'significant' in the 'significant to mankind' sense, what he says applies to the 'personally significant' just as well. To do significant things, we need to:

- Focus on the most important problems you can try to solve
- Work at those problems consistently
- Be excellent in our work

Not everyone can chose to work on they believe to be the Important Problems. Being able to do so is definitely a sort of privilege. However, there will always be problems that are the most important within your ability to solve. Hamming says to work on those.

You can apply this recursively. What's the most important piece of the important problem you're solving? Focus on that. It sounds simple, but is everyone you know working on the problem most important to them? Are you?

Being consistent, determined, driven -- that's harder to change. Everyone has some minimum they can do consistently. Hamming talks about finding that minimum, and then growing from there. Everyone can be consistent to a different degree, but we can all improve.

The final point is the one that resonated with me the most. Be excellent. So much of the field I work in is filled with bloat, workarounds, hacks, "good enough", and crumbling towers of complexity. The reality is that excellence isn't a prerequisite for generating profit. Excellence is, however, a prerequisite for doing significant things. This is the clearest goal, and yet the hardest to achieve. We all know what excellence looks like -- some even make a habit of excellence already. But it's easy to do mediocre work, and in some ways it's even easier to do good work. Excellent work is usually hard work.

Hamming concludes on the point of excellence. For him, the most important thing about the work we do is how we do it:

_"... it is nice to end up where you wanted to be, but the person you are when you get there is far more important. I believe a life in which you do not try to extend yourself regularly is not worth living—but it is up to you to pick the goals you believe are worth striving for."_
In Your Opinion, What Makes Code Pythonic?
Tags: watercooler, discuss, python
Published: 2020-05-27 23:37:22

#What does it mean, Pythonic?
What do you think makes code pythonic, write about it in the comments.
<img src="https://qph.fs.quoracdn.net/main-qimg-28cadbd02699c25a88e5c78d73c7babc">