teleren/devto huggingface dataset:
AWS StepFunction Distributed Map
Tags: aws, serverless, news
Published: 2022-12-08 15:57:12

The last day of re:Invent 2022 still had some nice talks about the newly released features. This one is an extension of the existing Map step that StepFunctions already has.

### Why

The existing Map step has some limitations regarding input/output size (256KB), logging the execution history (25000), parallelism (max 40). It is good enough to execute smaller tasks in parallel. If you want larger tasks you can of coarse use S3 as a storage and implement the loading and saving of data yourself in the Lambda function(s) but since this is a quit common pattern AWS decided the extend the functionality of the Map step.

### What

When using the new distributed processing mode you can specify a S3 source (either on file, objects with a prefix or the whole bucket). The internals of the Map step will then be treated a a separate workflow.

At the start of the execution of the parent StepFunction the information from S3 is gathered using ListObjectsV2 and then, using the parallellism and batching you specify, the child StepFunction is called. All attributes from ListObjectsV2 will be passed into the child StepFunction.

The maximum parallelism is increased up to 10000 parallel executions of the child StepFunction. The input and output limitations are no longer there. And because the inner StepFunction has its own execution history also this limitation is mitigated.

### Other sources than S3

Other sources are currently not supported. In order to use other sources you have to set up some steps in the parent StepFunction to prepare some S3 objects to use as input.

### Features

Because the amount of executions can be very large you can specify an error threshold. If the number of errors is lower then the threshold the Map step will still succeed.

In the Map step you can also specify an export location in S3. The results of running the Map step will be reported there (manifest.json, Succeeded.json, Failed.json, Pending.json). You can use a step after the Map step in the parent StepFunction to process these output files to the result you require.

In the console you can follow the progress of your distributed map step.

### Use cases

When doing map-reduce like operations on large sets of data this can be a very nice architecture. No need for spark or haddoop instances but a completely serverless solution.

To get a grasp of the size oof the workload you can process using this functionality. In the demo AWS presented a case where over 500000 S3 objects were processed in batches of 250 objects with a paralellism of 3000 parallel lambda executions, each lambda would take 16 seconds to process. The total execution time was under 2:30 minutes.

Kinesis streamed to S3 has a directory structure very suitable to process in this way. If the S3 objects are too large to handle in the lambda function you can use for example the AWS Lambda Powertools to have the lambda process consequtive parts of the objects.

Picture is from a hike we took on the day of arrival in Las Vegas. The location is called Valley of Fire.
React App Generator - Material Kit (VIDEO)
Tags: webdev, react, materialdesign
Published: 2022-12-08 15:46:07

---
title: React App Generator - Material Kit (VIDEO)
published: true
description: Learn how to generate a simple full-stack app on top of React & NodeJS using a modern Material Kit Design - Free Service
tags: webdev, react, materialdesign
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hoqrgq9votyisj4g5t7j.png
canonical_url: https://appseed.us/generator/react/material-kit/
---
Hello Coders! 

The VIDEO mentioned in this article explains [how to generate](https://appseed.us/generator/react/material-kit/) a simple full-stack product on top of **React** and **NodeJS** using `Material Kit` design for styling the pages. In the end, the source code is saved on GitHub under the MIT License (which can be used for commercial projects or eLearning). 

> `Thanks for reading!`

- 👉 [React App Generator](https://appseed.us/generator/react/material-kit/) - `Material Kit Design` (free service)
- 👉 Preview the [DEMO](https://react-material-kit.appseed-srv1.com/pages/authentication/sign-in) (from a similar project)

For newcomers, **Material Kit** is a popular design crafted and released for free by [Creative-Tim](https://www.creative-tim.com/?AFFILIATE=128200).  `Material Kit React` is built with over 40 frontend individual elements, like buttons, inputs, navbars, alerts, or cards, giving you the freedom of choosing and combining.

{% embed https://www.youtube.com/watch?v=S4yDfxsXNLA %}

---

This free **MUI & React Template** is coming with pre-built design blocks, so the development process is seamless, and switching from our pages to the real website is very easy to be done.

---

> `Thanks for reading!` For more resources and support, please access:

- 🚀 Free [support](https://appseed.us/support/) provided by AppSeed (email & Discord)
- 👉 More [free apps](https://appseed.us/apps/open-source/) crafted in Flask, Django, and [React](https://appseed.us/apps/react/)
Solving day 08 of Advent of Code 2022 in JavaScript
Tags: challenge, adventofcode, javascript, video
Published: 2022-12-08 15:42:40

---
title: Solving day 08 of Advent of Code 2022 in JavaScript
published: true
description: JavaScript walkthrough of Advent of Code 2022 (day 08)
tags: challenge, AdventOfCode, javascript, video
series: Advent of Code 2022 in Javascript
---

{% youtube tJFpBE5Afs0 %}
Crafting a Compiler in Rust: Introduction
Tags: rust, compilers, tutorial, computerscience
Published: 2022-12-08 15:40:16

Has anyone wondered? How does the code that we've written turn into a program or application? The simple explanation is that there's a program designed specifically to translate high-level code into machine code that computers can understand and execute. This program is called a compiler.

Hi, In this dev-blog series, I'm writing and documenting the process of creating a compiler for my programming language, Pernix. This series will go through to the point where the compiler is self-hosted, meaning it's possible to develop a compiler for my programming language using my programming language and compiler. This series will not be a line-by-line tutorial but will show what each component in the compiler does and how they interact.

## About my Programming Language and Compiler

The programming language is called **Pernix**. It is a Latin word; that means agile, quick, and swift. The language is statically-typed and compiled and has a syntax similar to C. I'll implement the compiler in Rust programming language with the help of **LLVM**. 

## Compiler Basics

Before writing a compiler, let's understand some basics of it. Generally speaking, the compiler is separated into the **front-end** and **back-end**. The front-end part involves converting the source code into an intermediate representation and checking all sorts of errors in the program. The back-end then performs the translation from the intermediate representation produced by the front-end into machine code.

Typically, each operating system and CPU have its own instruction sets, meaning that the compiler has to handle the differences between them.
Fortunately, **LLVM** will do the most heavy-lifting jobs in the back-end part. Therefore, I'll focus more on developing the front-end part.

### Compiler Front-End Overview

Again, the compiler front end can also be broken down into several parts:
**Lexical Analysis**, **Syntactic Analysis**, and **Semantic Analysis**.

- **Lexical Analysis** involves breaking the source code into manageable tokens/words.
- **Syntactic Analysis** performs the grammar check of the language and turns a token stream from the previous phase into the abstract syntax tree (AST).
- **Semantic Analysis** performs additional checks that the Syntactic Analysis phase can't, such as type-checking and symbols look-up. As the previous can only validate the grammatical structure of the program. 

![Brief Diagram of Compiler Front-End](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/26jr8xty0p5d2tdqn33z.png)

I'll go into more detail about each phase in their separate blog post of them.
Calling all talented developers AROUND THE GLOBE!
Tags: hiring, python, react, javascript
Published: 2022-12-08 15:40:09

**Remotebase** is Hiring Software Engineers! These are well paid, full time remote developer job opportunities. If you're looking to work with innovative Silicon Valley clients, become a part of a top 1% global developer team from the comfort of your home, then apply NOW : https://developers.remotebase.com?referral=pCL9jjjZp4T1Zk81VhYDxD ..take an online technical assessment (HackerRank*) and get an offer!
OSD Lab 9 - Setting up a CI Workflow
Published: 2022-11-19 04:01:20

This week, I set up a GitHub Actions CI Workflow to run tests automatically on any push to main or for a PR to main. I found that there was a [starter workflow file for rust](https://github.com/actions/starter-workflows/blob/main/ci/rust.yml) with this configuration. So, I simply added the workflow through the Actions tab using this starter file as it is. 

Next, I partnered up with [Wokneun No](https://github.com/Genne23v) to add more test cases to their [OpenSSG](https://github.com/Genne23v/OpenSSG) project written in Java. It had been a while since I had worked with Java, and I had never written any tests in Java. So, it took me some time to set up the project and get used to working with IntelliJ.
However, it was easy to understand the code and file structure. Unlike my rost_gen project which has all the logic in one file, it was well organized into multiple files. Though I had to run the tests using the IntelliJ IDE, writing the tests was way simpler.
I ended up going through a utils file and improved the existing tests by [adding more test cases for each method](https://github.com/Genne23v/OpenSSG/pull/18).

I had seen CI workflows before and had wanted to add it to my project for a while. They are great to ensure that tests are run for every change. I had been delaying it since I assumed it would be a somewhat complicated process to set it up. But it turned out to be quite simple.
How ChatGPT Will Help Creators To Optimize Social Media
Tags: ai, socialmedia, chatgpt, artificialintelligen
Published: 2022-12-08 21:07:58

---
title: How ChatGPT Will Help Creators To Optimize Social Media
published: true
date: 2022-12-08 15:24:37 UTC
tags: AI,socialmedia,chatGPT,artificialintelligen
canonical_url: https://guptadeepak.com/how-chatgpt-will-help-creators-to-optimize-social-media/
---

 ![How ChatGPT Will Help Creators To Optimize Social Media](https://images.unsplash.com/photo-1655720840699-67e72c0909d1?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fEFJfGVufDB8fHx8MTY3MDQ1NDk0Ng&ixlib=rb-4.0.3&q=80&w=2000)

[ChatGPT](https://openai.com/blog/chatgpt/) is an advanced artificial intelligence (AI) chatbot platform mimics human-like conversation based on user prompts. It uses natural language processing capabilities and machine learning algorithms to understand user conversations and engage with them in meaningful ways.

Many applications and use cases are still developing from a search engine, getting directions, automating customer support, automating product queries, etc. One use case for creators for ChatGPT is that it can analyze social media conversations and provide insights on the best ways to interact with followers, optimize posts for maximum engagement, and provide timely and helpful advice to users.

ChatGPT can help creators in the following ways to optimize their social media:

- **Recommend ideas:** It can identify topics likely to generate the most engagement and provide advice on how to post about those topics. 
- **Track and analyze conversations:** It can also track conversations and provide insights on the content that resonates with users. This can help creators understand the most successful content and tailor their strategies to focus on those topics.
- **Personalize recommendations:** It can provide personalized advice and recommendations for creators. This could include advice on when to post, what type of content to post, and how to engage with followers in meaningful ways. This personalized advice can help creators maximize their social media presence and ensure that their content reaches the right audiences.
- **Monitor potential issues:** ChatGPT can monitor conversations and identify potential problems or customer service opportunities. This can be especially useful for creators who have large numbers of followers and need to be able to respond quickly and effectively to customer inquiries or complaints.

ChatGPT is an invaluable tool for creators who want to optimize their social media presence. By leveraging its natural language processing capabilities and machine learning algorithms, creators can get personalized advice and insights on optimizing their content and engaging with their followers in meaningful ways. With ChatGPT, creators can maximize their social media presence and ensure that their content reaches the right audiences.
Sites that offer free certifications
Published: 2022-12-08 15:23:18

There are many websites that offer free certifications, often in the form of online courses or tutorials that users can complete to demonstrate their knowledge and skills in a particular subject. Some examples of websites that offer free certifications include:

Coursera: Coursera offers thousands of online courses in a wide range of subjects, many of which are free to take. Some courses also offer the option to purchase a paid certificate to verify completion of the course.

Khan Academy: Khan Academy offers free online courses and tutorials in subjects such as math, science, and history. While the courses do not offer formal certifications, users can earn badges and points to track their progress and achievements.

edX: edX is a nonprofit online learning platform that offers free courses and tutorials from top universities and institutions. Many courses offer the option to purchase a paid certificate to verify completion.

Udacity: Udacity offers free and paid online courses in subjects such as computer science, data science, and business. Some free courses offer a certificate of completion, while others require a paid subscription to access the certificate.

LinkedIn Learning: LinkedIn Learning (formerly Lynda.com) offers a wide range of online courses and tutorials, many of which are free to access. Some courses offer a certificate of completion for a fee, while others do not offer a certificate.

Overall, there are many websites that offer free certifications in various subjects, which can be a valuable resource for those looking to learn new skills and advance their careers. It's important to carefully research and compare different options to find the right fit for your needs and goals.
Stream
Tags: cheatsheet, java, stream
Published: 2022-11-26 11:07:33

Convert _n_ elements to an array list
```java
List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5, 6);
```

## Stream
Stream map
```java
List<String> listOfStrings = Arrays.asList("1", "2", "3", "4", "5");
 
List<Integer> listOfIntegers = listOfStrings.stream()
        .map(Integer::valueOf)
        .collect(Collectors.toList());
 
System.out.println(listOfIntegers);
```

Stream reduce
```java
//
// Example 1
//
List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5, 6);
int result = numbers
  .stream()
  .reduce(0, (subtotal, element) -> subtotal + element);
assertThat(result).isEqualTo(21);

//
// Example 2 (Use method reference)
//
int result = numbers.stream().reduce(0, Integer::sum);
assertThat(result).isEqualTo(21);

//
// Example 3
//
List<User> users = Arrays.asList(new User("John", 30), new User("Julie", 35));
int result = users.stream()
  .reduce(0, (partialAgeResult, user) -> partialAgeResult + user.getAge(), Integer::sum);
assertThat(result).isEqualTo(65);
```

Stream.flatMap
```java
//
// Example 1
//
Merging Lists into a Single List
List<Integer> list1 = Arrays.asList(1,2,3);
List<Integer> list2 = Arrays.asList(4,5,6);
List<Integer> list3 = Arrays.asList(7,8,9);
  
List<List<Integer>> listOfLists = Arrays.asList(list1, list2, list3);
 
List<Integer> listOfAllIntegers = listOfLists.stream()
          .flatMap(x -> x.stream())
          .collect(Collectors.toList());

System.out.println(listOfAllIntegers);

//
// Example 2
//
String[][] dataArray = new String[][]{{"a", "b"}, 
		{"c", "d"}, {"e", "f"}, {"g", "h"}};
         
List<String> listOfAllChars = Arrays.stream(dataArray)
              .flatMap(x -> Arrays.stream(x))
              .collect(Collectors.toList());

System.out.println(listOfAllChars);
```
---
## IntStream
```java
// Converting IntStream to Array
int[] intArray = IntStream.of(1, 2, 3, 4, 5).toArray();

// Converting IntStream to List
List<Integer> list = IntStream.of(1,2,3,4,5)
            .boxed()
            .collect(Collectors.toList());

// Returns stream containing a single specified element.
IntStream.of(10); // 10

// Returns stream containing specified all elements.
IntStream.of(1, 2, 3); // 1,2,3

// Returns a sequential ordered int stream excludes the last element.
IntStream.range(1, 5); // 1,2,3,4

// Returns a sequential ordered int stream includes the last element.
IntStream.rangeClosed(1, 5); // 1,2,3,4,5
```
AI-generated explanations and documentation for JavaScript code
Tags: javascript, ai, gpt3, typescript
Published: 2022-12-08 15:17:39

As a JavaScript developer, you know how challenging it can be to work with existing codebases. Often, the code is poorly documented, making it difficult to understand what it does and how it works. That's why I've added two new AI-powered features to the [JS Assistant](https://p42.ai?utm_medium=article&utm_source=devto) for Visual Studio Code: AI-generated JS-Doc comments and AI-generated explanations for code snippets.

### AI-generated JSDoc comments
The JS Assistant can [generate documentation for functions and methods](https://p42.ai/documentation/code-assist/document-function?utm_medium=article&utm_source=devto). This feature accelerates writing JSDoc comments by providing a pre-filled template you can refine.

![JS Assistant generates JSDoc comment](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6f80s6bjnl32ue2c2hwj.gif)

### AI-generated code explanations
You can [generate explanations of JavaScript code snippets](https://p42.ai/documentation/code-assist/explain-code?utm_medium=article&utm_source=devto) to help you understand complex or challenging code passages. While such AI-generated descriptions can sometimes be inaccurate or misleading, they can help accelerate making sense of a legacy codebase. They are best used with your code understanding, e.g., to provide ideas that get you started or unstuck.

![JS Assistant generates code explanation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vabwpkunnu4ftz8ah1bh.gif)

These AI-powered actions can help you work more effectively in complex legacy JavaScript code today. 

Code explanations and comment generation are early, experimental features. I would love to hear how you think they would be helpful?
What is Node.js?
Tags: javascript, node, beginners
Published: 2022-12-08 15:08:21


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b5siperpu8qkrvlul8ta.jpeg)

#### Overview
This article is **Part 8** of the series [Working With NodeJS](https://dev.to/meddy672/series/20844), a series of article to help simplify working with Node.js. In this article I will briefly cover what Node.js is and what it is not.

#### What is Node.js
Node.js is an open source cross-platform JavaScript runtime environment. It uses Chromes v8 engine, which is written in C++, to execute our JavaScript code in our applications on the platform of choice, as well as translate our JavaScript code into C++ code. Because chrome's v8 engine cannot interact with the filesystem or operating system, v8 must translate our JavaScript code so that C++ code can understand it which can interact with the filesystem and operating system. Node.js also uses the Libuv project, written in C, which provides The Event Loop. The Event Loop is like a scheduler that manages all asynchronous operations within our JavaScript program. When you start a Node.js program a single instance of The Event Loop is created and placed into a single thread. This means that our program runs on one core of the CPU. Node.js is not entirely single threaded. There are functions utilized within the Node.js library that run outside The Event Loop thus creating and using additional threads. You can manage the amount of threads available for these functions with the `process.env.UV_THREADPOOL_SIZE`. The default number of threads available is 4. 

### Resources

- [Advance Concepts In Node.js](https://www.udemy.com/course/advanced-node-for-developers/)
- [v8](https://v8.dev/docs)
- [Libuv](http://docs.libuv.org/en/v1.x/loop.html)
Undo git add command
Tags: git
Published: 2022-12-08 13:30:00

For adding any files for the commit we add them using `git add .` for adding all modified files or `git add ${fileName}` for adding one file at a time.

In some situations, you might want to undo this git add command. You can use the following command to do so

```
git reset
```

This command can be used to unstage all files that were staged for the commit.

Whereas if you want to unstage only one particular file you can use the following command

```
git restore --staged ${fileName}
```
Why Skills in Algorithmic Problem Solving are More Important Than Ever in 2023
Tags: computerscience
Published: 2022-12-08 15:07:15

As the field of artificial intelligence continues to advance, the importance of competitive programming is becoming increasingly clear. In the year 2023, the rise of AI systems like CoPilot will highlight the need for skilled programmers who can develop and maintain these complex systems.

Competitive programming, also known as algorithmic problem solving, involves writing computer programs to solve complex problems and puzzles. This type of programming requires a deep understanding of algorithms, data structures, and computer science principles, as well as the ability to think logically and solve problems efficiently.

In the age of AI, the importance of competitive programming cannot be overstated. AI systems like CoPilot, which is developed by OpenAI, are able to generate various kinds of code, from creating a server to making a game. These systems require highly skilled programmers to develop and maintain them, and the ability to solve complex algorithmic problems is essential.

Furthermore, the rise of AI is also creating new opportunities for competitive programmers. As AI systems become more advanced, there is a growing demand for skilled programmers who can work on these systems and help them reach their full potential. Competitive programming provides a valuable skill set that can be applied to a wide range of industries, from finance and healthcare to transportation and gaming.

In addition to the practical applications of competitive programming, the discipline also has numerous benefits for individuals. Competitive programming can improve problem-solving skills, logical thinking, and attention to detail, as well as provide a sense of accomplishment and achievement. It can also be a valuable tool for personal growth and development, as it encourages continuous learning and improvement.

In conclusion, the importance of competitive programming will only continue to grow in the age of AI. As AI systems like CoPilot become more advanced and widespread, the need for skilled programmers who can develop and maintain these systems will become increasingly clear. Competitive programming provides a valuable skill set that can be applied to a wide range of industries and offers numerous benefits for individuals.