teleren/devto huggingface dataset:
Access S3 public data without credentials
Tags: aws, cli, s3, nosignin
Published: 2022-04-04 06:19:09

# Ever in need of accessing publicly available S3 data ? 

Well, why don't we access them without our credentials, as they are already public & thus no need of credentials, right?

##What##
**AWS S3** 
Object storage solutions from AWS, which basically allows to store any data, structured, unstructured, videos, files, images any. We basically store data inside folders but it is just that they are called buckets in AWS S3 and that access permissions can be controlled at bucket levels

**AWS CLI**
Command Line Interface to access all available AWS Services from command line. We can use the CLI, if we just have credentials for user/SSO

##Why##
People or Teams working on huge volumes of data from Data science, Big Data, Machine Learning backgrounds would have & would be in need of such volume of data for their research & development. With developments in cloud, many such data sets are now available on cloud in AWS S3, and we will see about accessing them here

Publicly made available data are already made available for Public, and hence we can access/download them without having the need to supply AWS credentials

##How##

* Let us create sample s3 bucket and make it publicly available

1. First create a sample bucket(pick a unique name) 
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gm0fl7lgxcf4rwjn5f4x.png)
2. Now add couple of objects/files/images
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wf6sbmanizp2heii213d.png)
3. Make it public by unchecking the option "Block public access"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vfv1lvzkxohm7tc6psl2.png)
4. One last step is to give "Read" access to the public bucket for people to access. Sample policy here
```
{
    "Version": "",
    "Id": " ",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<<your-public-bucket>>"
        }
    ]
} 
```  

* Now let us access the same without providing credentials

1. Access to command prompt with AWS CLI installed in it  
2. Or, if possible, create or use an existing EC2 with AWS CLI for applying the s3 access command
```
aws s3 ls s3:<<your-public-bucket-name>> --no-sign-request
```
3. From the below screen shot, we can see the plain bucket access raising "Access Denied" but later works on using '--no-sign-request' option along with the cli command
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7obznhorfmohoplv7x1j.png)

Happy Public Data !!
Day 4
Tags: express, javascript, node
Published: 2022-04-04 06:19:00

---
title: Day 4
published: true
description: Make a simple login work
tags: express, JavaScript, NodeJS
//cover_image: https://direct_url_to_image.jpg
---

The idea is to make a simple login with express 

copy day3 folder into day4 folder

moving just contents from one folder into next folder in same root folder in terminal

> `cp -r folder1/* folder2`

create folder authenticate

> `mkdir authenticate`

Inside authenticate folder create new file login.js

> `touch login.js`

Paste this code

```
const login = function (user,password) {
    if(user==="brix" && password==="brix"){
        return true;
    }
    else{
        return false;
    }

}

module.exports = login
```
> ###Edit
> You have to add the two lines below to start working with json in express
> `app.use(express.json());`
> `app.use(express.urlencoded({ extended: false }));`

app.js should look more like this

```
const express = require('express')
const path = require('path')
const login = require('./authenticate/login')

const app = express()
const port = 3000

app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(express.static(path.join(__dirname, 'public')))

app.get('/',(req,res) => {
  res.sendFile(path.join(__dirname,'public/index.html'));
})

app.post('/login', function (req, res, next) {
    const user = req.body.username
    const pass = req.body.password
    let loginResult = login(user,pass)

    if(loginResult) {
      //show main content
      res.send('Hello World Again!')
    }else{
      //show error 
    }

})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
```
Am I late here?
Tags: firstpost, beginners
Published: 2022-04-04 06:03:11

I am a Full Stack Developer with more than 15 years of experience as a developer. My journey started with FoxPro 2.6 (Hell ya that's dos based tiny database software that was used as a programming language too), encouraging me to learn more.

Still I find myself newbie as I am late creating profile on dev.to
Running Webpack Dev Server with HTTPS on developer machine using MKCERT
Tags: webpack, https, mkcert
Published: 2022-04-04 05:51:23

1) Add local domain to `/etc/hosts`:

```
127.0.0.1 my-project.dev
```

2) Install [mkcert](https://mkcert.dev)
3) Create certificate for this domain:

```bash
➜ mkcert my-project.dev
   
Created a new certificate valid for the following names 📜
 - "my-project.dev"

The certificate is at "./my-project.dev.pem" and the key at "./my-project.dev-key.pem" ✅
```

4) Copy certificate `.dev.pem` and the key `.dev-key.pem` to project folder:

```bash
cp my-project.dev.pem path/to/my-project
cp my-project.dev-key.pem path/to/my-project
```

5) Add HTTPS options to Webpack Dev Server config:

```js
devServer: {
    host: 'my-project.dev',
    https: {
        key: fs.readFileSync('./my-project.dev-key.pem'),
        cert: fs.readFileSync('./my-project.dev.pem'),
    }
},
```

> Tested with:
> - `webpack 4.28.4`
> - `webpack-dev-server 3.2.1`
> - `@vue/cli-service 3.5.3`
> - `mkcert 1.3.0` from Homebrew
> - `macOS Mojave 10.14.6 (18G84)`
How many platforms do you use to market yourself and your work?
Tags: devjournal, beginners, todayilearned, discuss
Published: 2022-04-04 05:47:52

As I continue to post on dev.to, I consistently see post after post telling me to never stick to one site, one platform to publish my articles to.  Whether it be hashnode, Medium, or any other dev website, we are encouraged to broaden our scope to reach a wider audience. But how many is enough? And how many is too many?

Where else do you post? Do you count Instagram and Twitter? Or do you only stick to one platform? And why?
How to Add the Symbol Palette to an Angular Diagram Component
Tags: webdev, angular, productivity, diagram
Published: 2022-04-04 05:26:07

See how easily you can add and customize a symbol palette in the Syncfusion [Angular Diagram](https://www.syncfusion.com/angular-ui-components/angular-diagram) component. This video explains how to add a symbol palette to the Angular Diagram component and customize the palette header. You will see how to restrict expansion of the palette panel and customize the size of the symbols. 

Product Overview: https://www.syncfusion.com/angular-ui-components/angular-diagram

Explore tutorial video: https://www.syncfusion.com/tutorial-videos

Download a getting started example from GitHub: https://github.com/SyncfusionExamples

{% youtube rJAKtQEBq3Q %}
9 lessons a developer can learn from gym
Tags: programming, developer
Published: 2022-04-04 05:22:19

I started working out when I was in college. Almost at the same time, I started doing programming as well.

The similarities between both are astonishing.

Here are the lessons I have learned as a developer from gym.

You can read full article [here https://blog.canopas.com/9-lessons-a-developer-can-learn-from-gym-262fd817494b](here https://blog.canopas.com/9-lessons-a-developer-can-learn-from-gym-262fd817494b)
Two Sum Leetcode Solution Using Javascript
Published: 2022-04-04 05:05:15

```javascript
var twoSum = function(nums, target) { 
for (let i = 0; i < nums.length; i++) {
 for (let j = i + 1; j < nums.length; j++) {
 if (nums[i] + nums[j] == target) {
 return [i, j] 
} 
} 
} 
};

```
Insertion Sort - Python
Tags: insertionsort, sorting, python
Published: 2022-04-04 05:05:10


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uz1t04558lk7sm14cdo4.PNG)

```
def insertionSort(array):
    # Write your code here.
	for i in range(1,len(array)):
		j=i
		while j>0 and array[j]<array[j-1]:
			swap(j,j-1,array)
			j-=1
	return array		
def swap(i,j,array):
	array[i],array[j]=array[j],array[i]
    
#TC=O(n^2)  SC = O(1)  - Saipavan Seelamsetty
```
Steps to 👨🏿‍💻👨🏿‍💻👨🏿‍💻 pt 1
Published: 2022-04-04 04:48:44

#Building New Projects 

![Late Night Coding](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yr2zfws50sq2y4nl9z83.jpeg)

I'm in the process of not just working on building my javascript skills. But also attempting to build my first game. While I don't expect this to become a console classic. I want to create an approach that could be used not just for this project but for others in the future.

One of the first thing I did was create a roadmap. While I know that it seems obvious. I wanted to approach this with learning goals and a flexible direction.

#Notes.md file
Inside of this file I created a list of things I wanted to accomplish with this project. 
1. What was the goal of this app? 
2. What files I would need?
3. What am I testing for?
4. What are my stretch goal? 

#Steps To The Goal 
![steps](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6tfw8qnqhd34cimjqvtp.jpeg)

#Pseudo Code
![pseudo code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1gofdjpi82ymtdvnrbuw.png) 

Pseudo Code is a way of mapping out steps to help you solve a problem or describe a line of code. 

Up to this point. I have written out what each function is doing and what each file is for to help me process through each step. 

## Test Driven Development
![Test Driven Development](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zi3oa395szzsdr02eze2.jpeg)

Test Driven-Development is a process of repeatedly testing software against different edge cases throughout the process. 

While building out this game so far. I've been mindful to test everything and check my notes.md file to make sure everything is where I want it and to make updates as needed.

If something is not working or passing the test. The next question is why? 

Review your code its important to make sure you go through it with a fine tooth comb? 
One of the biggest things that I have experienced so far is spelling.

The saying dot your eyes and cross your t's is an under statement. 


Hopefully this outline of my process helps you on your journey. Look forward to sharing the journey with you and I complete this process. 

Happy Coding 👨🏿‍💻👨🏻‍💻🧑🏾‍💻👩‍💻
DaemonSets - Why, when and how
Tags: kubernetes, beginners, tutorial
Published: 2022-04-04 04:40:07

##### Why & When
The basic unit of deployment in kubernetes is a `Pod` and a managed objected with guaranteed number of replicas is a `Deployment`. In-case of a deployment the nodes on which pods land-up is decided by the scheduler based on the scheduling policy.

**Use DaemonSet when**
- When you want one pod of your application to run on each node (or identified subset of nodes)
- When you want to address those pods directly and not through the load-balancer

**Typical use-cases**
1. Running agents on kubernetes worker nodes - monitoring agents
2. Running aggregators on a per node basis - log aggregators
3. Running node-wise daemon processes for distributing load - [Interesting usecase from this dev.to post](https://dev.to/azure/use-daemonset-as-a-cache-when-referencing-large-files-from-each-pod-11lb)

---
##### How
Creation
```bash
❯ cat /tmp/ds.yaml
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd-ds
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: fluentd-ds
  template:
    metadata:
      labels:
        name: fluentd-ds
    spec:
      tolerations:
        # this toleration is to have the daemonset runnable on master nodes
        # remove it if your masters can't run pods
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      containers:
        - name: fluentd-ds
          image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2

❯ kubectl create -f /tmp/ds.yaml
daemonset.apps/fluentd-ds created

❯ kubectl get daemonsets.apps/fluentd-ds -n kube-system
NAME         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
fluentd-ds   4         4         4       4            4           <none>          116s

❯ kubectl get pods -n kube-system -o wide | grep fluentd-ds
fluentd-ds-mm26b                                1/1     Running   0          2m45s   10.244.0.6   macbook-control-plane   <none>           <none>
fluentd-ds-nx9hb                                1/1     Running   0          2m45s   10.244.3.4   macbook-worker2         <none>           <none>
fluentd-ds-tfrvn                                1/1     Running   0          2m45s   10.244.2.3   macbook-worker          <none>           <none>
fluentd-ds-vv5rx                                1/1     Running   0          2m45s   10.244.1.4   macbook-worker3         <none>           <none>

```

Communication
All usual modes of services - ClusterIP, NodePort, LoadBalancer and Headless service work (though ClusterIP and LoadBalancer does not make too much sense). General norm is to use `ports.hostPort` and then map it through a `NodePort` and address via `<node-ip>:<node-port>`
```yaml
ports:
- containerPort: 24224
  hostPort: 24224
  name: "fluentd-port"
```

---
##### More info
- [In-depth talk by "That DevOps Guy"](https://www.youtube.com/watch?v=RGSeeN-o-kQ&t=9s)
- [Related GitHub link](https://github.com/marcel-dempers/docker-development-youtube-series/tree/master/kubernetes/daemonsets)
Swift - Structure and Class
Published: 2022-04-04 04:22:53

- Both Structure and Class can define property, **method, subscript, initializers,** can be **extended** and conform to a **protocol**.

- Features only class have but structure don't have - **Inheritance, TypeCasting, De-initializer, Reference Counting** (allows more than one reference to a class instance)

example : 

``` swift
struct Resolution {
    var width = 0
    var height = 0
}
class VideoMode {
    var resolution = Resolution()
    var interlaced = false
    var frameRate = 0.0
    var name: String?
}
```

``` swift
let someResolution = Resolution()
let someVideoMode = VideoMode()
```

- Memberwise Initializers for Structure Types - All structures have an **automatically** generated **memberwise initializer**, which you can use to initialize the member properties of new structure instances. 

- Structures and Enumerations Are **Value** Types
A value type is a type whose value is copied when it’s assigned to a variable or constant, or when it’s passed to a function.

**_Integers, floating-point numbers, Booleans, strings, arrays and dictionaries—are value types in swift, and are implemented as structures behind the scenes._**

- Classes Are Reference Types. Reference types are not copied when they’re assigned to a variable or constant, or when they’re passed to a function. 

- classes are **reference** types, it’s possible for multiple constants and variables to refer to the same single instance of a class. **Identity operator** is used to check whether two constants or variables refer to the same single instance.
  
  1. Identical to (===)
  2.   Not identical to (!==)
Jest and recurring actions
Tags: testing
Published: 2022-04-04 04:07:34

Sometimes you want to have Jest perform a recurring action between each task.

Some examples:
Query a database, clear storage, clear mocked data, or reset a mocked route.

We don't really want to be bothered with having this recurring code in each test, and luckily Jest has a solution for us.

## Recurring actions

There are four functions we can hook into:

- `beforeEach`: Runs before each test
- `afterEach`: Runs after each test
- `beforeAll`: Runs before all tests
- `afterAll`: Runs after all tests

Let's sketch an example.
We have a database function to call, so the steps we want to achieve are:

- create database
- populate with mock data
- re-populate for each test
- remove database

This scenario is a perfect case for all four functions to hook into.

The first thing we want is to create our database, for which we'll use the `beforeAll` function.

```js
beforeAll(() => {
  return createDatabase();
});
```

The next step is to populate the database with demo data we can alter in our tests.

```js
beforeEach(() => {
  return populateDatabase();
});
```

Our test might alter/remove/create elements in this database, so we want to clear it between each test.

```js
afterEach(() => {
  return clearDatabase();
});
```

And once we are all done, we should remove the database so the next run will be fresh again.

```js
afterAll(() => {
  return removeDatabase();
});
```

And that's it, these four steps will now run at the needed times.
To showcase this, let's create this sample test file and see when each call is used.

```js
test('user database has Chris', () => {
  expect(db.user.hasName('Chris')).toBeTruthy();
});

test('user database doesnt have Thomas', () => {
  expect(db.user.hasName('Thomas')).not.toBeTruthy();
});
```

The firing order is as follows:

- `beforeAll`: Created the database
- `beforeEach`: Populates database
- Test 1 runs: Find user Chris
- `afterEach`: Clear database
- `beforeEach`: Populates database
- Test 2 runs: Can't find Thomas
- `afterEach`: Clear database
- `afterAll`: Remove database

And that's the flow it will take.

We can quickly make our test more manageable and work in specific ways to ensure each test is solid and fresh.

### Thank you for reading, and let's connect!

Thank you for reading my blog. Feel free to subscribe to my email newsletter and connect on [Facebook](https://www.facebook.com/DailyDevTipsBlog) or [Twitter](https://twitter.com/DailyDevTips1)
Ways of fixing flaky tests in Cypress
Tags: cypres, cypressi, qaautomation
Published: 2022-04-04 04:04:38

Ways of fixing flaky tests in Cypress

https://kailash-pathak.medium.com/ways-of-fixing-flaky-tests-in-cypress-840329d759e5

https://www.youtube.com/channel/UCrRPMnWk2slXl4xFOVXfTmQ?sub_confirmation=1
Project showcase: Learnfluent
Tags: javascript, html, showdev, beginners
Published: 2022-04-04 03:28:43

This was the first real web project I ever built. I thought it would be cool to show it off.

This is a Chrome extension I created a few years ago. It is designed to help people learn a new language. Each day you are given a new word from the language of your choice. Currently French, Spanish, German and Russian are the language options. There are 30 words total and one is shown each day. 

More info on how to get started on Chrome extension development: [https://developer.chrome.com/docs/extensions/mv3/getstarted/](https://developer.chrome.com/docs/extensions/mv3/getstarted/)
Link to source code: [https://github.com/fig781/chromeExtensionWordADay](https://github.com/fig781/chromeExtensionWordADay)
Link to download: [https://chrome.google.com/webstore/detail/learnfluent/diffhlcafbihhmocdaknniaddfjefigh](https://chrome.google.com/webstore/detail/learnfluent/diffhlcafbihhmocdaknniaddfjefigh)
How to add unit test cases to your project using Jest
Tags: javascript, webdev, beginners
Published: 2022-04-04 03:07:15

A without test cases is just a text. Textcases are important for any application they makes the web app more efficieny and most importantly all the edge cases will be covesred in these test cases. Now that we are talking about testing lets talk about a testing framework called JEST for Javascript. 

There are some other really great frameworks out there, and choosing one comes down to flavor and needs.

For this article, I'll use a simple JavaScript project as our basis.It will have a determination script to determine if certain statements are true or false.

## Check any function 

You can see the determine.js function in the main folder.
This is basically our app. It's used to determine if a variable is a number.

In the most basic case, it should perform the following tests:

```
isNumber(123);
// Should be true

isNumber('abc');
// Should be false

```
We first have to add Jest to our project to make this testable.

```
npm install -D jest
```
This will install the Jest package in our dev dependenies. Now we should modify our `package.json` to include a testing command. 

```
"scripts": {
    "test": "jest"
}
```

The cool thing about Jest is that it can fire automatically on any files it deems testable.

By default, it can run on the following files:

- .js files inside __tests__ folders
- .test.js files inside your project
- .spec.js files inside your project

For now, let's add a `determine.test.js` file. This makes sense as we have our `determine.js` file as our basis.

To start the file, we have to import our function first.

```
const { isNumber } = require('./determine');
```
Then we can define tests. Each test is an isolated scope that can pass or fail.

Let's start by adding a test made to expect the right answer.

```
test('Validate a number', () => {
  expect(isNumber(1)).toBeTruthy();
});
```

We start by defining the test, which holds a string (the name of the test) and executes a function.
The actual test states:

Expect -> function (variables) -> to be true

I really love how Jest made these super human-readable and understandable.

We ask for the function isNumber to be true when we pass the number one to it.

We can now run the test by invoking the following command:

```
npm run test
```
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hnhiwj2mbwm5c8g23br1.png)

Wow, the test is succeeding. Now lets add a failing test case. 

```
test('Invalidate a string', () => {
  expect(isNumber('ABC')).toBeTruthy();
});
```
We use the same test but pass a wrong number, so the test should fail.
 ![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j5eaf19hw9wemudw1uor.png)
 
And it does fail, which is expected!
So let's modify the test case to evaluate a false value.

#Conclusion 
Although this is a very straight forward test, it can be super important for example in any given number form field you need a numerical value and not any string. You can use this test case there. Moreover its in your hand how to make test cases more complicated. 

##Thank You
An easy way to group objects in java
Tags: java, map, group, programming
Published: 2022-04-04 03:06:51

In this example, we will be making grouping easy to use on Maps objects that contain collections as values.
For instance, we hava a `Map` of `Integers`, that has its values as an `ArrayList` of `Strings` : 

```java
Grouper<Integer, String> grouper = new ArrayListGrouper<>();
        grouper.put(1, "a");
        grouper.put(1, "b");
        grouper.put(1, "c");
        grouper.put(1, "c");
        grouper.put(2, "c");
```
The output will be : 
```shell
{1=[a, b, c, c], 2=[c]}
``` 
All we have to do, is to define a grouping strategy, for example the already defined `ArrayListGrouper` class uses an `ArrayList` as its strategy.
We can always define a new Grouper that will use another 
 `GroupingStrateg`.

Let's change the `ArrayList` to a `HashSet`, so that the elements become unique : 
```java 
public class HashSetGrouper<K, V> extends Grouper<K, V> {

    public HashSetGrouper() {
        super(HashSet::new);
    }
}
```
Testing it like : 
```java
@Test
    public void testHashSetGrouper() {
        Grouper<Integer, String> grouper = new HashSetGrouper<>();
        grouper.put(1, "a");
        grouper.put(1, "b");
        grouper.put(1, "c");
        grouper.put(1, "c");
        grouper.put(2, "c");
        System.out.println(grouper);

    }
```
The output will become : 
```shell
{1=[a, b, c], 2=[c]}
``` 
The `1` key has now a set in which the `"c"` value is not repeated.

the code is hosted on Github : https://github.com/BelmoMusta/java-Grouper
Your feedback is welcomed.