teleren/devto huggingface dataset:
Scanning multipage documents without duplex mode (but with feeder scanner) on linux (crossmerge pdf files)
Tags: linux, bash, tutorial, scanning
Published: 2022-01-16 23:47:40

If one wants to digitalise multi page documents but just has a feeder scanner that does not support duplex scanning: This is for you.

Having small documents that are printed on both pages is a pain to scan into one single file but it is possible.
E.G. one could scan each individual page and combine everything using `pdfunite` or `pdftk` or any other tool that supports joining pdf documents.
If the document one has are 20 or more pages. This method is very time consuming.

If one has a feeder scanner, here is what one could do:

## Terminology
In this tutorial I will refer with `A` as the front side of a sheet and `B` will be the back. In addition numbers will refer to a specific sheet number.
e.G. `4B` will be the backside of the 4th sheet (= page 8) 

## HowTo

1. One puts all the pages into the feeder scanner and scan each `A` side of each document. The result will be a file with the pages `1A 2A 3A... nA`. We will call this file `front_pages.pdf`
2. Now one turns the stash of pages around so that the last page is on top. One puts the pages into the feeder scanner and scans the back pages of each sheet. The result will be a document containing every `B` side of each sheet in reverse order `Bn, Bn-1, Bn-2, ..., B3, B2, B1`. We will call this file `back_pages_reverse.pdf`

The last thing to do is to merge the pages. Because `back_pages_reverse.pdf` is in reverse order one must merge the documents in the way:
- front_pages[1]
- back_pages_reverse[n]
- front_pages[2]
- back_pages_reverse[n-1]

...
- front_pages[n-1]
- back_pages_reverse[2]
- front_pages[n]
- back_pages_reverse[1]

This can be done with a simple bash script:
Prerequisites: The script uses `pdftk`. So to use it one must install `pdftk` first.

Example to install `pdftk` on fedora or debian based systems
```bash
#fedora
sudo dnf install pdftk

#debian
sudo apt-get install pdftk
```

Script for creating the meged pdf file

crossmergereverse.sh
```bash
#!/bin/bash

numPages=$(pdftk $1 dump_data | grep NumberOfPages | awk '{print $2}')

param=""

for ((i=1 ; i <= $numPages ; i++ ));
do
	bindex=$(($numPages-$i+1))
	param="$param A$i B$bindex"
done

pdftk A=$1 B=$2 cat $param output $3
```
Running the script like

```
sh crossmergereserve.sh front_pages.pdf back_pages_reverse.pdf completeDocument.pdf
```
Will result in a newly created document `completeDocument.pdf` with all the pages in the correct order `1A 1B 2A 2B... nA, nB`

The script `crossmergereserve.sh` can be found [here](https://github.com/typedcode/crossmerge). The repository also contains a script for a simple cross-merge of pdf documents.

## Why `back_pages_reverse.pdf`?
The script for merging the documents would be simpler if there would be two documents like `1A, 2A, ..., nA` and `1B, 2B, ..., nB`. But to achieve that one must reorder the entire document before scanning the back sides. Things one has to do manually are are less if one does it that way.
An Introduction to Open Web Components
Published: 2022-01-16 23:45:14

JavaScript is used widely across the internet, as it is natively run in your web browser. As a web developer, familiarizing yourself with JavaScript is vital to success. Open-wc streamlines aspects of development and utilizes Node.js and npm. From the open-wc Git Repository's README: "Open Web Components provides a set of defaults, recommendations and tools to help facilitate your web component project."

What do Node.js and npm have to do with JavaScript? Node.js is a cross-platform JavaScript run-time environment that executes JavaScript code outside of a browser. The Node Package Manager, or npm, is a software registry of over 800,000 packages installed alongside Node.js according to w3schools. Node.js is one of the most popular tools for open-source developers to work with JavaScript. In this article, I will walk through the steps required to start using open-wc on a Windows machine.

## Getting Started
First things first, you will need an IDE to work with JavaScript. You may use any IDE you are comfortable with if it is compatible with JavaScript, but I will be using Visual Studio Code which can be downloaded [here](https://code.visualstudio.com/).

## Installing Node.js and npm

![Download page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qkk0ponf32aod6a3gqwl.png)
To install Node.js, go [here](https://nodejs.org/en/download/). Get the most current LTS version for your operating system for the best stability, or the latest current version for new features at the cost of some stability. Since npm is included with the Node.js install, you should now have both; open your terminal and enter `node -v` to check if Node.js was installed, and `npm -v` to check if npm was installed. You're all set to continue if you see version numbers after running those commands.

![Check if Node.js and npm are installed](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ivrqvddmt4b436pqlh5u.png)

## Starting a new open-wc project
To begin with open-wc, make a directory to put your project in. Change your terminal directory to this new location, and enter `npm init @open-wc`. If everything has been installed correctly, this should run and you can now begin working on your open-wc project. I initialized my project with these options:

- New project scaffold
- Web component
- Linting, Testing, and Demoing enabled
- No TypeScript
- Install using npm

![open-wc initialization](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/045vzmnqn7t8awe2qwo5.png)

The hello-world element is located in the `/src/` folder to be edited further. And with that, you have just created your first web component!


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/00r10js53b1gxr0pfr5c.png)
HAXCMS in Docker
Published: 2022-01-16 23:42:49

# What is Docker

Docker is a platform as a service(PaaS) product that containerizes applications with only necessary dependencies. As a result, no matter what OS runs the container, it will behave the same. Using a tool such as Kubernetes allows for multiple containers and thereby instances of the same application to be deployed at once. Docker allows containers to be tagged allowed for versioning and additional metadata to be saved. In production environments, docker is used to create containers but Kubernetes is used to deploy containers.

# How to Install Docker for Mac
1. Go to [https://www.docker.com/products/docker-desktop](https://www.docker.com/products/docker-desktop) and download Docker Desktop 
2. Move Docker Desktop to the Applications Folder

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lgsvdd3fcw6ekwl8pzzh.png)
 
3. Close the Installer
4. Open Docker Desktop
         _You may get the following prompt, just select open and continue_
         ![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/68qmx0g77d3acz7sxnxv.png)
 
5. Create a Docker account and sign in.


# Building an Image and Running Container
1. Go the HAXCMS [GitHub repo](https://github.com/elmsln/haxcms) and download it.
2. Open VSCode to the repo or terminal and navigate to the directory
3. Build the docker repo and name it by using the following command `docker build . -t <Insert Wanted Name Here>`
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lsx4xs12wdjy4krajt4b.png) 
4. Run the container by using the 'docker run -dp 80:80 <Name of Image>'
5. Open browser to `localhost:80`
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5duc4ybqatt4rjpvjt1r.png)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mhos6tmy02fh2yvbt7rw.png)
  

# How to stop container
1. Open Terminal
2. Run `docker ps`
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xiyo49754h6h2c2nrleb.png) 
3. Run `docker stop <containerID>`
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/at5vf99anpx416gyp9tf.png)
Building Basic Web Components 
Tags: webcomponents, microsoft, beginners, tutorial
Published: 2022-01-16 23:32:42

Before starting this class I had a minuscule amount of exposure to web development, JavaScript, and HTML. After this lab I can now confidently say that investing in a device that uses a Linux platform is the wave for anybody pursuing a career in web development. 

## SOFTWARE REQUIRED
Before starting anything, if you are on windows, I highly recommend installing the Windows Terminal found in the Microsoft store, it's a free application that will allow you to add a Linux distro and access its terminal freely. The reason to do this is because "ohmyzsh" doesn't like windows users and makes you jump through multiple loopholes to get it working correctly on your bill gates powered OS.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hrge09gdx57fgwyjt07f.png)
 
When setting up this terminal, the first step you must take is by enabling "Windows subsystems for Linux" found in "Windows Features" (This can be found in your computers search bar) After this install a Linux distro such as Ubuntu 
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k4gp0ujmhsuoo2slz7s2.png)
 add it to your terminal (may require restarting both programs) and enjoy the benefits of having your Linux terminal run through windows such as

1. The ability to now copy and paste without an external program
2. Ohmyzsh now works by copy and pasting the link to install from the website into your terminal

**NODE.js**
Node.js is an "asynchronous event JavaScript runtime" and its basic functionality is to allow users to build and develop scalable network applications. 
**NPM**
NPM is a package manager that comes installed with Node.js that enables a user to discover, install, and use node programs.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5tqb8868cjfl2itlzwds.PNG) What it should look like when these programs are properly installed.

## The Lab

Once all the required components are installed to your system, you can proceed with the lab

1. Proceed to create and open a new folder using the `mkdir` command and `cd` command respectively 
2. Run the command `npm init @open-wc ` and if all the previous steps were done correctly, the prompt for open web components will pop up. 
3. Select all of the options using "a", don't use typescript, and tag your web component as hello world. Install your dependencies with either Yarn or NPM and the you will be given a new pop up in browser.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ff6m9z64q3lfzqfckgl6.png)
4.  The page you will be taken to willbe a mianly blank page with hello world at the top and a button that lets you increment the number ![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uarml3j5ouqnqw29mlwt.png)
 
Congratulations you have created your very own web component.
2021, favorite resources
Tags: healthydebate
Published: 2022-01-16 22:54:53

This last year I've not been very active working on dev.to.
2021 brought me a huge opportunity to work on a full-remote, English speaking company with a high level of technical base, where I can be mentored in new skills (Golang, TDD, Hexagonal Architecture) while putting my 15+ year knowledge into good use.

So, will skip to a brief annual content insight of 'favorite resources' and 'a year in images'

--- 

# Open Source Projects

> _Freedom of source, Freedom of usage, Freedom of enhancement.
Let's thank back those that provide these helpful tools._


![Love back](https://media.giphy.com/media/HJmz8M2YsimLm/giphy.gif)


- [Diagrams as Code](https://diagrams.mingrammer.com/) based on [graphviz](https://graphviz.gitlab.io); for prototyping cloud system architectures 

- [DirEnv](https://direnv.net); unclutter your .profile 

- [Cloud Custodian](https://github.com/cloud-custodian/cloud-custodian); rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources 

--- 

# Tools

## cli :: wormhole

```
$ whatis wormhole
wormhole (1)- Securely and simply transfer data between computers
```

```
-- will give you a autogenerated_code for the receiver to 
$ wormhole send whatever_file

$ wormhole receive autogenerated_code
```
 
aweeesomeeee
![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n2z3a53mso3x9e1xuxpy.png)
 
edit: also in web version in case no CLI is available https://wormhole.app

## cli :: pre-commit

> A framework for managing and maintaining multi-language pre-commit hooks. 

https://pre-commit.com/

## Firefox Containers

A must when you need to deal with several cloud-accounts at the same time :sweat:

https://www.thechiefmeat.com/guides/containers.html

## cli :: checkinstall

avoid using make install in favour of chekcinstall


> CheckInstall keeps track of all files installed by a "make install" or equivalent, creates a Slackware, RPM, or Debian package with those files, and adds it to the installed packages database, allowing for easy package removal or distribution. 

https://help.ubuntu.com/community/CheckInstall

--- 


# Other

- [10 Software Engineering Laws Everybody Loves to Ignore](
https://twitter.com/jschauma/status/1359344687287439362)

- [Remote-work at gitlab](https://learn.gitlab.com/coursera-remote-work)

---

What's been your most relevant resources this year? Let's share those #dirtylittlethings
-
Tags: webdev, beginners, javascript, programming
Published: 2022-01-16 23:25:16

Mi primer post en esta red social, no lo niego, me siento curioso por comenzar el viaje como escritor de tecnología y además compartir mi viaje como ingeniero de software.

Llevo dos años como desarrollador de software para web, así es como me gano la vida en este momento. 
Gracias a esa oportunidad me he dado cuenta del sesgo tecnológico en la entidad donde radico es inmensa. 

Es por esa razón que voy a iniciar mi camino en compartir conocimiento y experiencias en el mundo tech. Organizaré mis tiempos para poder generar el contenido necesario para lograr la mejor experiencia. 




Emocionado por la experiencia. 
-
**Jorge Cornejo**
I erroneously deletedy root partition. Now what?
Tags: archlinux, linux, grub, help
Published: 2022-01-16 23:25:09

Posted on Reddit's `r/archlinux` and `r/techsupport`.



Is there a way I can reinstall Arch from the bootstick I still have?  

I've been following [this](https://www.diskpart.com/articles/grub-rescue-unknown-file-system-0825.html#toc.0.40649366859536595) tutorial, and how my system and it differs is that all of my partitions have "unknown filesystems".



Here's a backlog:  
```

    error: disk `lvmid/MMwt86-jYqe-hUn1 ... zo7KQw' not found.
    Entering rescue mode...
    grub rescue> ls
    (hd0) (hd1) (hd1,gpt2) (hd1,gpt1)
    grub rescue> ls (hd0)
    (hd0): File-system is unknown.
    grub rescue> set
    cmdpath=(hd1,gpt1)/EFI/grub_uefi
    prefix=(lvmid/MMwt86-jYqe ... zo7KQw)/boot/grub
    root=lvmid/MMwt86-jYqe-hUn1-x1eI-sqCk-RoXD-NihFx4/49zeY2-wEvE-jsN6-2EqG-5TW4-CeLE-zo7KQe
    grub rescue>

```

And the same error I get for `(hd0)` applies to the other three (`(hd1)`, `(hd1,gpt2)`, and `(hd1,gpt1)`).

IMPORTANT NOTE:
`(hd1,)` is my bootstick. Removing it beforehand results in `(hd0) (hd0,gpt2) (hd0,gpt1)`.
JavaScript Project
Tags: javascript
Published: 2022-01-14 17:34:42

I finished my phase 4 project for Flatiron School! I created a single-page web application for my JavaScript project.

I created 2 separate repos for the [front end](https://github.com/victoriaehrbar/book_review_frontend) and [back end](https://github.com/victoriaehrbar/book_review_backend). The back end was fairly similar to the phase 3 Rails project, with controllers, models, and has_many relationships. 

My project was a book review/tracker, where a user can input a title, author, description, and category/genre. A category has_many books and a book belongs_to a category.

While it was interesting to create a mini Rails project and review some old material, I was most interested in the front end JavaScript portion and refining what I learned in phase 4. I utilized the fetch method to get the book data to display on the page, display new posts, and to delete posts. 

The back end part of the project required creating a Rails API, which was interesting as the phase 3 Rails project did not incorporate this. The front end JavaScript portion of this project communicates with the API for the front end and back end to work together.

In JavaScript, event listeners are used to create actions. For example, my project has an attachDeleteButtonListener, which lets the server know that when the delete button is clicked, that book entry should be removed from the page. There is also another event listener that lets the page know to create a new book when "submit" is clicked on the new book form.

Overall, the JavaScript project was a great fusion of Rails and JavaScript that allowed me to synthesize my JavaScript knowledge and create a fun single-page web application.
The Problem With API Monitoring
Published: 2022-01-16 23:03:17

Often times developers are proud of their newly created, shiny and beautiful APIs. But what if they don't have the resources to analyze the real-life usage of their software? How will they ever be aware of bottlenecks and other problems in their API if they don's set up proper monitoring?

The issue with monitoring is however, that it is often not trivial to set up, as it might require you to build completely new infrastructure or modify your existing one or even implement your monitoring from scratch.

The other important aspect is pricing. The big players on the monitoring field are not cheap. You might be fine starting with some of the most common solutions out there, like DataDog or New Relic.

As your API usage grows however, you will notice that your monitoring costs will increase significantly with the traditional solutions. This is where you might consider either changing your vendor or building your own solution.

See full example on how to implement an API analytics solution for your API at: [https://www.apilytics.io/blog/problem-with-api-monitoring](https://www.apilytics.io/blog/problem-with-api-monitoring)
My Capstone Project: Deep Learning to Detect Bugs in Code using Graph Based Neural Networks
Tags: deeplearning, python, dotnet
Published: 2022-01-16 23:01:05

Over this long weekend, I decided to revisit details of my capstone project for my Master's in Data Science (graduated May 2020) using Deep Learning to detect bugs in code. The paper can be found [here](https://drive.google.com/file/d/1x5G8eKDVW8MTHLEE585I_UTOC4idNmoO/view?usp=sharing). 

This paper involved conducting a comparative study of the efficacy (based on test accuracy) of Gated Graph Neural Networks (GGNNs) vs. Relational Graph Convolutional Networks (RGCNs) on a task to automatically detect the misuse of a variable for the top 25 trending C# repositories on Github.

The results showed that RGCNs outperformed GGNNs for all cases (did a considerable amount of randomized hyperparameter tuning but wasn't fruitful to shake up the results), albeit, within < 5%.

My work was based on multiple papers by Microsoft Research (particularly https://lnkd.in/gd5kTEEv) and used Tensorflow to conduct the analysis. In a nutshell, the training data obtained was a modified version of the Abstract Syntax Tree generated by the Roslyn compiler.

Some lessons I picked up from this experience are:

1. Sticking with a white paper even if you don't understand any of it in the beginning.
2. Digesting material through different media is a good to way to switch it up: I heavily relied on YouTube videos of Deep Learning conferences (https://lnkd.in/gzcTRGhG) to get a more lecture based approach to bolster my learning.
3. Treating data as a first class citizen. If possible, using version control / backing up data is a definitely a lesson I learnt through this. Clobbering my old weights was something I did more times than I like to admit.
4. I am a big advocate of self describing code, however, since this was such a new space for me with a steep learning curve, commenting as much as I could made a significant difference.
5. Testing on small byte sized chunks saved me a considerable amount of time down the road: prototypical testing saved me countless hours because of the countless runtime errors that I could have faced if I hadn't done an E2E run.

Any feedback would be greatly appreciated! Happy to answer any questions.
2021, a year in images
Tags: healthydebate, motivation, challenge, quiz
Published: 2022-01-16 22:54:55

This last year I've not been very active working on dev.to.
2021 brought me a huge opportunity to work on a full-remote, English speaking company with a high level of technical base, where I can be mentored in new skills (Golang, TDD, Hexagonal Architecture) while putting my 15+ year knowledge into good use.

So, will skip to a brief annual content insight of 'favorite resources' and 'a year in images'

---

> _Some people show their vacation photos, why not show our work-time experiences with screenshots? :D_


I've been collecting screenshots in my work-time for a while and I think they represent, in some way, the technology I worked with, and my technical career

---

- tmate screen resolution hell

started pairing with my colleagues and having a two-screen setup vs. a huge screen can give them some pain when we're tmate-ing :sweat: sorry! xD
tried to find ways of fixing this, but didn't find any solution for us... drop me a line if you got any suggestion

![tmate does not resize well](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yq7ok8ha4luud7l6ayhq.png)
  

- tech people, we've never been good at selling stuff

![git - the stupid content tracker](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i9zvuuvssvverw7etyza.png)


- our robot thinks you're a robot!

this is happening more and more...  [getting into ridiculous situations](https://www.reddit.com/r/ProgrammerHumor/comments/rv17yn/this_not_a_joke_this_is_real_skype_account)

![apps still think i'm a robot - soundcloud ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b9fd2hav2gj1q6vliut6.png)


- Onedrive loop hell

hilarious situations when using OneDrive [during my previous job](https://dev.to/wideawakening/agile-the-choice-of-tools-aka-a-linux-user-working-on-a-windows-stack-based-company-27oc).
![onedrive loop hell 1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vkvk9tzzfrxajbh4rgyo.png)
![onedrive loop hell 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w6zm203b7wsxsaw5xro4.png)


- non-transparent duck duck go results

still going for ducky as the first search engine options, but sometimes it gives funny results where i wonder why am i seeing those results :confused:

![duck duck go search result for zerolog plain text log](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6bhj9sy833mya3a4r2je.png)

![google search result for zerolog plain text log](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l3dnyducyp8whke4dj6i.png)

to add some value, resolved how to do this during our daily mob session :)
```
zerolog.NewConsoleWriter()
```