How to handle large files with R using {readr} 2.0
Channel: Bruno Rodrigues
Date: 2021-05-16

In this video I show you how I compute over a 13 gigs text file using the new development version of {readr}.

As of filming (16-05-2021) {readr} 2.0 is in development. If you want to install it, following the instructions here (but first check if {readr} 2.0 hasn't been already released on CRAN!):

https://twitter.com/jimhester_/status/1392198233036836865?s=20

Script to the video: 

https://gist.github.com/b-rodrigues/afd96186311117c858fdf6cd6cfa8184

Another package that you might be interested in is {disk.frame}, which solves the issue of reading, and working with, medium size data that does not fit in memory:

https://diskframe.com/index.html

I also wrote a blog post about it:

https://www.brodrigues.co/blog/2019-09-03-disk_frame/

Follow me on twitter: https://twitter.com/brodriguesco​​​​
Read my blog: https://www.brodrigues.co/​
Find these videos useful? You can buy me a coffee to express your gratitude: https://www.buymeacoffee.com/brodriguesco
Learn R by reading my free ebook: https://b-rodrigues.github.io/modern_R/