teleren/devto huggingface dataset:
Superfast CSV imports using PostgreSQL's COPY command
Tags: postgres, database, backend, rails
Published: 2017-09-07 07:49:27

---
title: Superfast CSV imports using PostgreSQL's COPY command
published: true
description: A Rails-oriented look at the options for importing large CSV files, how slow it is via an ORM, and how the COPY command can help.
tags: #postgresql #databases #backend #rails
canonical_url: https://infinum.co/the-capsized-eight/superfast-csv-imports-using-postgresqls-copy
cover_image: https://thepracticaldev.s3.amazonaws.com/i/6q7wi936xujbicjzclbn.png
---

_This post was originally published on [The Capsized Eight Blog](https://infinum.co/the-capsized-eight/superfast-csv-imports-using-postgresqls-copy)_

Dealing with various sources of data in web applications requires us to create services that will extract information from CSV, Excel, and other file types. In that case, it's best to use some existing libraries, or if your backend is on Rails, use gems. There are many gems with very cool features like [`CSVImporter`](https://github.com/pcreux/csv-importer) and [`Roo`](https://github.com/roo-rb/roo). But you can also use plain [`Ruby CSV`](http://ruby-doc.org/stdlib-2.0.0/libdoc/csv/rdoc/CSV.html).

Either way, if those are small CSV files, you will get your job done easily. But what if you need to import large CSV files (~100MB / ~1M rows)? 

![Postgres COPY](https://thepracticaldev.s3.amazonaws.com/i/r5mjhydxlvwtb4j5x3i7.png)

I found that PostgreSQL has a really powerful yet very simple command called [`COPY`](https://www.postgresql.org/docs/9.6/static/sql-copy.html) which copies data between a file and a database table.
It can be used in both ways:

- **to import data from a CSV file to database**
- **to export data from a database table to a CSV file**.

Example of usage:

```sql
COPY forecasts
FROM 'tmp/forecast.csv'
CSV HEADER;
```

This piece of SQL code will import the content from a CSV file to our `forecasts` table. Note one thing: it's assumed that **the number and order of columns in the table is the same as in the CSV file**.

### Results

Importing a CSV file with ~1M rows now takes **under 4 seconds** which is blazing fast when compared to previous solutions! 

For more details, read [the original blogpost](https://infinum.co/the-capsized-eight/superfast-csv-imports-using-postgresqls-copy).
Let's use LaTeX!
Tags: vscode, latex, tutorial
Published: 2017-09-07 02:01:57

---
title: Let's use LaTeX!
published: true
description: Quick how-to for installing and using LaTeX
cover_image: https://thepracticaldev.s3.amazonaws.com/i/v0ceqcyty1m6pbyz4jdp.jpg
tags: vscode, latex, tutorial
canonical_url: https://nickymarino.com/2017/09/07/lets-use-latex/
---

_This post was originally posted on my blog [here](http://nickymarino.com/2017/09/07/lets-use-latex/)._

[LaTeX](https://www.latex-project.org/) is a beautiful documentation system. It's similar to Markdown, but has many more features and is commonly used for academic papers and other publications that require a lot of equations. In this quick how to, we cover how to install LaTeX and use Visual Studio Code as an editor/previewer.

# Installing LaTeX

I recommend downloading [TeX Live](https://www.tug.org/texlive/) for Windows/Linux, and [MacTex](https://www.tug.org/mactex/) for macOS.
# Setting up our editor

If you haven't already, install [Visual Studio Code](https://code.visualstudio.com/) and go through a [tutorial](https://code.visualstudio.com/docs/introvideos/basics). Then, we need to install our extension for LaTeX itself. Head over to [LaTeX Workshop](https://marketplace.visualstudio.com/items?itemName=James-Yu.latex-workshop) and click install. 

# Using LaTeX

Now that we have our editor setup, we can write our first project. All LaTeX documents have a (non-blank) file that ends with `.tex`, which is the "main" file that has all of the text of the document. Since LaTeX usually generates more files (such as .log, etc.) while building the document, it's recommended that every document you want to write has its own folder.

For starters, create a file named `example.tex`:

```
\documentclass{article}
    % General document formatting
    \usepackage[margin=0.7in]{geometry}
    \usepackage[parfill]{parskip}
    \usepackage[utf8]{inputenc}

\begin{document}
% Put your text here
This is starter text.
\end{document}
```

Press `Ctrl-Alt-B` to build your project (or use the Command Palette), then `Ctrl-Alt-T` to view the pdf in a new tab. The end result should look like this:

![picture](https://thepracticaldev.s3.amazonaws.com/i/h7udmn31hzf8cdc3zcwb.PNG)

# Conclusion

LaTeX and VSCode are a great combination that you can use to write beautiful reports and papers. Check out a [tutorial](https://www.latex-tutorial.com/tutorials/) or [two](http://www.rpi.edu/dept/arc/training/latex/class-slides-pc.pdf) to realize the full experience LaTeX has to offer.

*Edit: [Fanny](https://dev.to/fannyvieira) recommends another [great tutorial](https://github.com/LewisVo/Begin-Latex-in-minutes) as well.*
*Edit 2: Fixed a tutorial link.*
Hi, I'm Josh Oiknine
Tags: introductions, php, javascript
Published: 2017-09-07 02:00:41

---
title: Hi, I'm Josh Oiknine
published: true
description: 
tags: introductions, PHP, javascript
---

My last name is pronounced [/wa:kni:n/](https://en.wikipedia.org/wiki/Initial_Teaching_Alphabet)

I have been coding for 14 years professionally

You can find me on Twitter as [@joshOiknine](https://twitter.com/joshoiknine)

I currently live in Philadelphia, PA but I was born and raised in the San Fernando Valley (Los Angeles, CA)

I work remotely for a new company yet to be named

I mostly program in these languages: PHP, JS

I am currently learning more about Angular, React

Nice to meet you =)
Who are your favorite speakers?
Tags: discuss
Published: 2017-09-06 18:31:00

---
title: Who are your favorite speakers?
published: true
description: Who are your favorite speakers?
tags: discuss
---


Hi there

We're currently in the process of planning a developer summit for our team and looking for a speaker to come spend some time with us. Unfortunately for me, I'm having trouble thinking of some names so I'd like to hear from the community who some of your favorite speakers are.

We're hoping to have someone give a talk on some more general development (language agnostic) topics like writing clean code, working with teams, or even in specific areas of technology like how and why to use the blockchain.

All suggestions welcomed!

Thanks!
We created the Crystal language, ask us anything!
Tags: ama, crystal
Published: 2017-09-06 17:23:20

---
title: We created the Crystal language, ask us anything!
published: true
description: Crystal is a programming language that aims to be friendly for both humans and computers alike.
tags: ama, crystal
cover_image: https://thepracticaldev.s3.amazonaws.com/i/ot7xpwy1j2bg74qebl49.gif
---

Hi there!

I'm Matias Garcia Isaia, and together with [Brian Cardiff](@bcardiff) and [Martin Verzilli](@mverzilli), we're part of the Crystal Team working at Manas.Tech.

[Manas.Tech](https://manas.tech/) is a software company from Buenos Aires, Argentina, that develops unconventional tech projects for people around the world. And, after being involved in Crystal's development for a while, I can assure you there are not that many conventional things in developing a programming language!

[Crystal](https://crystal-lang.org/) is a programming language that aims to be friendly for both humans and computers alike - make developers enjoy writing code, and make code run as efficiently as it can. Statically typed, compiled language with a really heavy type inference to make it feel as scripting - the best of both worlds. And did I say [it's open source?](https://github.com/crystal-lang/crystal)

Its first commit has [just turned 5](https://github.com/crystal-lang/crystal/commit/220bb8c736381d5beda5e7bc36c8d7f0d59badf6), and it has changed from an individual's hobby back then into an amazing language supported by Manas.Tech and [an amazing community](https://crystal-lang.org/community). To celebrate that, we are doing this AMA. Starting today, at 2PM EST, we'll do our best to share with you everything we know about the project, life, the universe, and everything! 

So - don't be shy, ask us anything!
Problems with Flask and PyCharm
Published: 2017-11-15 11:36:23

---
title: "Problems with Flask and PyCharm"
published: true
tags: 
canonical_url: https://kurzgedanke.de/post/problems-with-flask-and-pycharm/
---

I started a new Flask project and coded it in VisualCode and a Terminal. Therefore, I set up my virtual environment, installed Flask and started to code after the [Flask Quistart](http://flask.pocoo.org/docs/0.12/quickstart/).

The code looks like this:

```
from flask import Flask
app = Flask( __name__ )

@app.route('/')
def hello_world():
    return 'Hello, World!'
```

Then I ran the app with:

```
$ export FLASK_APP=hello.py
$ flask run
 * Running on http://127.0.0.1:5000/
```

Because of the great support for Web Apps I decided to switch to PyCharm. I imported the project into PyCharm, set the Interpreter to the virtualenv, but when it hit run nothing happend….

If you have the same problem: Feel welcome, I’ve got the solution!

Create a New Project, choose the Flask Template and select your existing flask project folder. Say `yes`to the pop-up which asks you to create a project from existing source.

![PyCharm Project Creation](https://kurzgedanke.de/img/flaskPyCharm/newProject.png)

Your project should open now and you can change your intepreter in the settings to your virtualenv or whatever you desire.

If you try to run the app now you should see something like this:

![PyCharm Console with nothing in it](https://kurzgedanke.de/img/flaskPyCharm/nothing.png)

I tried everything but the solution is damn simple… add this at the end of your code:

```
if __name__ == ' __main__':
    app.run()
```

and your console should output `* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)`.

If you want to know more about the `if __name__ == ' __main__':` line I can recommend [this Video from Corey Schafer](https://www.youtube.com/watch?v=sugvnHA7ElY&feature=youtu.be).

Thank you for reading!
It's not just someone else's computer
Tags: rant
Published: 2017-09-23 22:09:54

---
title: "It's not just someone else's computer"
published: true
tags: Rant
canonical_url: https://blog.jonstodle.com/its-not-just-someone-elses-computer/
---

I have to take a stand against the expression which have become so popular:

> "The cloud is just someone else's computer."

First off; it might be a revelation to some people, who aren't very technical, but have now suddenly discovered cloud computing, that _The Cloud_ is actually built on top of a lot of servers, which again is a highly specialized computers. That might feel like it's "just someone else's computer."

**But. For any technically inclined person, this is not an excuse.**

Saying it's "just someone else's computer", is like saying that a taxi is "just someone else's car."

Apart from the server itself, there is so much else that's done for you in the cloud. There's maintenance, repairs, upgrades and replacement. You don't have to invest in a new server, you only pay for what you use.

If you're using software as a service, even updates and patches to the operating system, the software itself and supporting systems are handled in the background.

There's an infrastructure which is kept up to date. And that infrastructure is connected to the backbone of the internet. It means that instead of having your server trudge through all the proverbial small country roads before getting to the highway, it's directly connected to the large highway instead.

There is so much work and orchestration going on in the background that makes it so very, very much more than someone else's computer.

/rant



_This post was originally published on [blog.jonstodle.com](https://blog.jonstodle.com/its-not-just-someone-elses-computer/)_
How to save on AWS Elastic Beanstalk EC2 machines by putting them to sleep
Tags: aws, devops
Published: 2017-09-06 07:02:44

---
title: How to save on AWS Elastic Beanstalk EC2 machines by putting them to sleep
published: true
description: How I optimized cost of AWS Elastic Beanstalk by suspending machines during the off work period
cover_image: https://thepracticaldev.s3.amazonaws.com/i/31nrbzijwflo6n49h67r.png
tags: aws, devops
---

If you are running few non production environments on AWS Elastic Beanstalk you were probably wondering how do you put its servers to sleep during the off work hours just to save few 💰💰💰?

There is a simple trick that AWS provide, but don’t promote it for a reason. As a result not many know about it. ;)

All you need to do is to define two time periods in your environment *Configuration -> Scaling -> Time-based scaling section.*

![](https://cdn-images-1.medium.com/max/800/1*KlB0ZzhaW4OoihmK5J0ALQ.png)

One period with cron based time settings to Stop your EC2 environment machines and the other one to Start them.

Here is cron definition to reduce number of EC2 machines to zero at 17:00 UTC every working week day — *0 17 * * 0,1,2,3,4*

![](https://cdn-images-1.medium.com/max/800/1*-lCFhTldteYTZyPGaN9gNQ.png)

And a cron definition to increase number of EC2 machines to one at 05:00 UTC every working week day — *0 5 * * 0,1,2,3,4*

![](https://cdn-images-1.medium.com/max/800/1*7OpOwEJ-3Bu09VoYRucjIA.png)

What it actually does is terminates and recreates the machines as AWS doesn’t allow to suspend/pause EC2 machines.

If you can afford to run without Elastic Load Balancer it will save you even more as this trick won’t terminate the LB. 🤑
Cocoapods: What It Is And How To Install?
Tags: swift, objectivec, xcode
Published: 2020-09-22 10:28:48

---
title: Cocoapods: What It Is And How To Install?
published: true
date: 2017-09-06 00:00:00 UTC
tags: Swift,ObjectiveC,XCode
canonical_url: https://www.loginradius.com/engineering/blog/what-is-cocoapods-and-installation/
---

Cocoapod is a package dependency manager for Objective-C and Swift projects with XCode. Those Languages are used to make applications that run iOS, macOS, watchOS and tvOS. Cocoapods also acts as a coding hub to share your code with other people. That way we as a community of…

   
[Read On](https://www.loginradius.com/engineering/blog/what-is-cocoapods-and-installation/)
EOL End of Line or Newline Characters
Tags: engineering, eol, linux
Published: 2020-09-03 05:01:09

---
title: EOL End of Line or Newline Characters
published: true
date: 2017-09-06 00:00:00 UTC
tags: Engineering,EOL,Linux
canonical_url: https://www.loginradius.com/engineering/blog/eol-end-of-line-or-newline-characters/
---

Which character do you consider as the end of line or newline? Most developers will answer \n (except for front-end developers, they would say: "tag" :) ). But this is not true, let's understand why. What is an End of Line character: It is a character in a string which…

   
[Read On](https://www.loginradius.com/engineering/blog/eol-end-of-line-or-newline-characters/)
Welcome Thread - v3
Tags: welcome
Published: 2017-09-05 22:06:38

---
title: Welcome Thread - v3
published: true
description: Introduce yourself to the community!
tags: welcome
---

Hey there! Welcome to dev.to!

![International Welcome Taco](https://media3.giphy.com/media/l2JHZ0dIcyFo5UQGQ/giphy.gif)

These welcome threads just keep going! And that's cause there's just more and more of you!

Please introduce yourself to the community! You can start by sharing your favorite programming languages, what you're interested in learning next, or your latest and greatest side project. No rules, just make yourself at home! ðŸ˜€

Also, here's a random question to get you started:

> **How has technology changed how you connect with others?**

Leave a comment below to introduce yourself to the community!âœŒï¸
Code Reviews That Don’t Suck w/ Vaidehi Joshi
Tags: techtalks
Published: 2017-09-05 20:12:06



---
title: "Code Reviews That Don’t Suck w/ Vaidehi Joshi"
published: true
description: ”Learn how to improve code reviews through changing team behavioral norms.”
cover_image: 
tags: #techtalks
---


_Great talk that I thought would benefit the DEV community._


{% youtube lge345hJKJk %}

### Main Argument

 “I love code reviews in theory. In practice, they are only as good as the group that's responsible for conducting them in the right manner.”

This quote, eloquently sums up how many of us feel about code reviews.The major thing I took away from this talk, is that to have better code reviews we need to proactively build a healthier developer culture/behaviors around them.

According to [Vaidehi](https://dev.to/vaidehijoshi) the largest factors that contribute to better code reviews are  social. Reviewing everyone pushes for a culture that values vulnerability. It's important for other developers to see that even senior developers can make mistakes and that it's ok. She advises us to develop empathy by also highlighting what was done well. Also, code reviews don't have to be rooted in negatives, but instead in shared ownership and responsibility of what we’re building together. And finally, that we need to initiate more conversations about code reviews, whether you feel your team code review process isn't where it needs to be or about what's working (if you're proud of it).




### Let’s Discuss


How do you currently practice code reviews? What does this version do well or what could it do better?
Using RabbitMQ in lead generation system development
Tags: appdev, rabbitmq, technology
Published: 2017-09-05 16:33:31

---
layout: post
title: "Using RabbitMQ in lead generation system development"
published: true
tags: appdev, rabbitmq, technology
cover_image: "https://thepracticaldev.s3.amazonaws.com/i/fsbi8sha0vw1jevpr57l.jpg"
---


In this post, you'll find a practical example of applying RabbitMQ in software development.

</span><a href="https://www.rabbitmq.com/">RabbitMQ server</span></a><span style="font-weight: 400;"></span></a><span style="font-weight: 400;"> is a queue server which allows different applications to interact with each other according to AMQP protocol, widely used by such corporations as Google, Microsoft, NASA, Red Hat, VMware, AT&T.
 
</span><a href="https://www.rabbitmq.com/getstarted.html">RabbitMQ</span></a><span style="font-weight: 400;">  is employed for developing service architecture when the following selected services process resource intensive and dormant tasks:
 
Xero – service for online bookkeeping
Secure Trading – service for payment processing
 
RabbitMQ is used to establish communication between servers; perhaps, it is one of the most efficient solutions for this purpose. 

Imagine, that the company has the following problem: with an increasing number of clients, a lot of time was spent on such financial service operations as invoice creation, payment request, payment verification, and confirmation (the operations are made manually).


<b>Tasks:</b> 

To automate processes of creating invoices via third-party Xero API and delivering invoices via email; make automatic debiting from user card via Secure Trading API.
 
 
<b>Solution with cron</b> 

The first pancake was a failure. We created a script, which runs two times a week and made necessary actions within a single process. This imposed constraints on the overall performance (long answer waits from API). Also, any mistake on one of the stages led to finishing the work of the whole process and required a complete script restart.

So, we came to the conclusion that we needed to divide code into several independent processes with the ability of communication between them.


Learn more about using </span><a href="https://smartum.pro/blog/using-rabbitmq-in-building-the-system-of-leads-generation/">RabbitMQ in building a lead generation system</span></a><span style="font-weight: 400;"> and <a href="https://smartym.pro/blog/using-rabbitmq-message-broker-for-messenger-development-some-clues-for-developers/">creating messengers</a>.

Also, feel free to share your experience in working with this technology!)
The 7 Most Popular DEV Posts from the Past Week
Tags: top7
Published: 2017-09-05 15:02:06

---
title: The 7 Most Popular DEV Posts from the Past Week
published: true
description: A round up of the most-read and most-loved contributions from the community this past week.
tags: icymi
cover_image: https://thepracticaldev.s3.amazonaws.com/i/xsr1z6iqv0pd7p3s6ns3.png
---

Every new week we round up some of the last week's top posts, comments, and tweets. If you have any feedback, please leave a comment._ :heart:

#1. Is working remote as glamorous as they say?

According to [Peter Angela](https://dev.to/peteranglea), it's a mixed bag. He started working remote six months ago, and now he shares his tips for those lured by the glamour. 

{% link https://dev.to/peteranglea/6-months-of-working-remotely-taught-me-a-thing-orten %}

#2. AMA with Stephanie Hurlburt and Rich Geldreich, texture compression experts

We had our first duo AMA on dev.to where we had the opportunity to _ask anything_ to the folks behind [Binomial](http://www.binomial.info/). They both work on Basis, a texture compression product that's about to become industry standard. We learned lots of things from how they deal with anxiety to how they decided texture compression could be a viable business. 

{% link https://dev.to/sehurlburt/were-stephanie-hurlburt-and-rich-geldreich-ask-us-anything %}

#3. Reading list for software engineers

Our author, [Lorenzo](https://dev.to/lpasqualis), has over twenty years of software engineering experiences. These are the eleven books that he recommends.

{% link https://dev.to/lpasqualis/11-books-all-software-engineers-must-read %}

#4. Programming paradigms, dissected

In this article, Eric Normand shares his take on procedural programming, object oriented programming, and functional programming. He describes their features, methodologies, and paradoxes. 

{% link https://dev.to/ericnormand/programming-paradigms-and-the-procedural-paradox %}

#5. But I want to have fun!
 
We all agree that deep down, software development is a lot of fun. Here's an analysis of how all our fun gets ruined. 

{% link https://dev.to/theobendixson/how-to-suck-the-fun-right-out-of-software-development %}

#6. Write good code, a guide for all

While targeted for beginners, this is a guide for all developers looking to write 'good code.'

{% link https://dev.to/isaacandsuch/steps-to-better-code %}

#7. Step one: escape from the cave

Here's a list of five things to strive for, if you want to be an awesome programmer.

{% link https://dev.to/theodesp/how-to-be-an-awesome-programmer %}

<img src="https://media.giphy.com/media/l1J9skVOajpIYloyc/giphy.gif" alt="lauren-lapkus-meme">


_That's it for our weekly wrapup! Keep an eye on dev.to this week for daily content and discussions...and if you miss anything, we'll be sure to recap it next Monday!_