teleren/devto huggingface dataset:
Filtering Googlebot IPs from a list of IP addresses
Tags: bash, googlebot, script
Published: 2018-06-25 15:05:10

---
title: Filtering Googlebot IPs from a list of IP addresses
published: true
description: 
tags: bash, googlebot, script
---

Hey there dev.to community!

I recently needed a simple script to filter Googlebot IPs from a list of IP addresses to be able to extract actual Googlebot visits from an access log.

Thankfully, Google provides [a method to make sure a visitor is actually Googlebot](https://support.google.com/webmasters/answer/80553?hl=en).

If you have a similar need, there you go:

```bash
#/bin/bash
#
# Performs reverse and forward DNS lookups to list Googlebot's IPs, given a list
# of IP addresses as a file. Useful for filtering access logs to find out actual
# Googlebot visits.
#
# An implementation of https://support.google.com/webmasters/answer/80553?hl=en

while IFS='' read -r IP_ADDRESS || [[ -n "$IP_ADDRESS" ]];
do
    IS_GOOGLEBOT=0
    REVERSE_LOOKUP="$(host $IP_ADDRESS)"
    echo "$REVERSE_LOOKUP" | grep -E "google.com.$|googlebot.com.$" > /dev/null && IS_GOOGLEBOT=1

    if [[ IS_GOOGLEBOT -eq 1 ]]; then
        FORWARD_LOOKUP="$(host $(echo "$REVERSE_LOOKUP" | cut -d " " -f 5) | cut -d " " -f 4)"
        if [[ "$FORWARD_LOOKUP" = "$IP_ADDRESS" ]];
        then
            echo $IP_ADDRESS
        fi
    fi
done < "$1"
```

You may save it as something like `filter-googlebot-ips.sh` and provide a file with a list of IP addresses to filter (each on a single line), as an argument. Like so:

```shell
$ ./filter-googlebot-ips.sh access-log-ips.txt > googlebot-ips.txt
```

This will perform reverse and forward DNS lookups for each of the IP addresses and print out the verified Googlebot IPs to `STDOUT`, which you can write to a file like in the example above.

Hope it helps someone out there! 🙌

*PS: [Here is a GitHub Gist](https://gist.github.com/merty/57a4c272afbc38a0a79128f718a80f62) if you prefer that.*
The 7 Most Popular DEV Posts from the Past Week
Tags: top7
Published: 2018-06-25 14:06:38

---
title: The 7 Most Popular DEV Posts from the Past Week
published: true
description: A round up of the most-read and most-loved contributions from the community this past week.
tags: icymi
cover_image: https://thepracticaldev.s3.amazonaws.com/i/sfwcvweirpf2qka2lg2b.png
---

#1. Totally Obsessed

A thoroughly thought-out method to git commits. 

{% link https://dev.to/shreyasminocha/how-i-do-my-git-commits-34d %}

#2. "Not math again!"

If math has ever stopped you from finishing a programming book...remember, they're just fancy words and symbols. The rabbit hole is not as scary as it may seem.

{% link https://dev.to/acoh3n/that-programming-book-you-never-finished-3oeb %}

#3. Breaking it down

A helpful guide for both the creator and review of pull requests. What are PRs, how to effectively create a PR, how to give feedback on PRs and how to respond to feedback.

{% link https://dev.to/kulkarniankita9/the-art-of-humanizing-pull-requests-prs-2238 %}

#4. All the brackets

A cheat sheet for all the different ways to use brackets in Bash. Because there are a lot of brackets with a lot of meanings.

{% link https://dev.to/rpalo/bash-brackets-quick-reference-4eh6 %}

#5. Channel your inner Sindre Sohrus

A walkthrough on building CLI apps: piping to the command line, dealing with sequential actions, executable JavaScript files, adding package binaries, and publishing to npm. 

{% link https://dev.to/hugo__df/how-to-make-beautiful-simple-cli-apps-with-node-c1b %}

#6. Did I do unit tests wrong?

K shares their experience with unit tests that leads to a community-wide discussions around the topic. 

{% link https://dev.to/kayis/what-are-the-alternatives-to-unit-tests-2jii %}

#7. Basic concept & Features

Iren shares an overview of Sails.js, a framework that makes extensive use of code generators to get your next app started with writing less code. 

{% link https://dev.to/iriskatastic/sailsjs-is-a-nodejs-mvc-framework-inspired-by-ruby-on-rails-23cp %}

_That's it for our weekly wrap up! Keep an eye on dev.to this week for daily content and discussions...and if you miss anything, we'll be sure to recap it next Monday!_
I'm an open source enthusiast at Mapbox, the creator of Leaflet and 40+ other JS libraries, and a rock musician. AMA!
Tags: ama, opensource, javascript
Published: 2018-06-25 13:56:06

---
title: I'm an open source enthusiast at Mapbox, the creator of Leaflet and 40+ other JS libraries, and a rock musician. AMA!
published: true
description: 
cover_image: https://thepracticaldev.s3.amazonaws.com/i/lyy3xr9x5g2nrcaddifn.jpg
tags: ama, opensource, javascript
---

Hey! I'm Vladimir Agafonkin (aka [@mourner](https://twitter.com/mourner)) from Kyiv, Ukraine.

In 2008, I started working on a small web maps library in secret from my boss, who insisted that I should stop "reinventing the wheel" and just use "mature, established solutions". When I reached out to the map developers community, my idea was ridiculed for "wasting time instead of contributing to existing projects".

This little library, [Leaflet](https://leafletjs.com/), eventually became one of the most influential projects in the mapping industry. It is used by Facebook, GitHub, Foursquare, Evernote, Flickr, Pinterest, Etsy, Craigslist, European Commission, NPR, The Washington Post and the like, and has almost 22k stars and 600 contributors [on GitHub](https://github.com/Leaflet/Leaflet).

For the last 4.5 years, I've been building mapping tools of the future at [Mapbox](https://www.mapbox.com/), focusing on the next-generation map technology, [vector maps](https://www.mapbox.com/maps/), and in particular [performance](https://twitter.com/mbostock/status/976847549503074305), data visualization and computational geometry algorithms. Here are a few examples of what I do:

- [A dive into spatial search algorithms](https://blog.mapbox.com/a-dive-into-spatial-search-algorithms-ebd0c5e39d2a)
- [How I built a wind map with WebGL](https://blog.mapbox.com/how-i-built-a-wind-map-with-webgl-b63022b5537f)
- [Introducing heatmaps in Mapbox GL](https://blog.mapbox.com/introducing-heatmaps-in-mapbox-gl-js-71355ada9e6c)
- [Rendering big geodata on the fly with GeoJSON-VT](https://blog.mapbox.com/rendering-big-geodata-on-the-fly-with-geojson-vt-4e4d2a5dd1f2)

To keep track of all the open source projects I'm involved with, I created [this list](https://github.com/mourner/projects). There are many, thanks to Mapbox — I'm incredibly lucky and grateful to have the privilege to work on open source full-time, and do it remotely from Kyiv.

Outside of engineering, I write songs, play guitar and sing in progressive rock band [Obiymy Doschu](http://rain.in.ua/en.html). We've recently released an album we've worked on for 8 years. The lyrics are Ukrainian, but [give the album a chance](http://rain.in.ua/son/en.html) — if you like beautiful, evocative, multilayered rock music, it will speak to you nevertheless.

I'm also a happy father of 4-year-old twin girls. [Here's me and my family](https://thepracticaldev.s3.amazonaws.com/i/p5wdez4frp2tvjjw9e7o.jpg) celebrating Ukraine's Independence Day last year.

Ask me anything!
Who's looking for open source contributors? (June 25 edition)
Tags: discuss, opensource
Published: 2018-06-25 13:45:52

---
title: Who's looking for open source contributors? (June 25 edition)
published: true
description: Please shamelessly promote your project. Everyone who posted in previous weeks is welcome back this week, as always.
tags: discuss, opensource
---

Please shamelessly promote your project. Everyone who posted in previous weeks is welcome back this week, as always. 😄
Ser Developer, como el comercial nos ve
Tags: spanish, mexico, personalopinion, teams
Published: 2018-06-25 13:32:45

---
title: Ser Developer, como el comercial nos ve
published: true
description: ¿que tan diferentes somos? ¿qué podemos aprender de ellos?
tags: spanish, mexico, personal opinion,  teams
---

![Developer.jpeg](https://thepracticaldev.s3.amazonaws.com/i/2agepg2y2zb560ro2kcu.jpeg)

A todos nos ha pasado que no podemos entendernos con una persona no-técnica así que evitamos convivir mucho con ellos. Pero, ¿por qué las personas de áreas comerciales como marketing, ventas o atención a clientes también nos evitan?

Realicé dos preguntas a personas que trabajan con developers. Las preguntas fueron relacionadas a la personalidad:

* ¿Qué es lo que más te intriga la personalidad del developer?
* ¿Qué cambiarías?

Lamentablemente, ambas preguntas coinciden en cuatro puntos: en su mayoría, somos introvertidos, lógicos, cuadrados y con un alto ego.

> **Somos Introvertidos y cuadrados**

Es muy común ver que el developer de la empresa no se involucra con el resto del equipo, se excluye o no tiene deseo alguno de convivir. Supongo que puede ser parte de nuestro perfil, pero ¿cuadrados?

Raramente vemos algo como tentativo o probable en una propuesta, creemos que sólo existe 0 o 1. A mi parecer, esto es causa de la educación que recibimos en la universidad ya que tenemos que suponer todos los escenarios dejando de creer que nada es aleatorio y limitandonos a no pensar *out of the box*.

> **Nos creemos superiores**

Esta es la respuesta que más me desconcertó, ¿realmente tenemos un ego elevado?

Como mencioné en mi post anterior, durante la carrera siempre (o casi siempre) nos dijeron que hay que facilitarle la vida al usuario, literalmente tuve profesores que decían el usuario es un idiota (lo cual está muy mal). En mi perspectiva, antes de cambiarme a un rol comercial creía que los demás (el usuario) no tenían mi misma capacidad intelectual cuando esto nunca fue cierto y creo que varios developers tienen la misma creencia.

Quizá no tengan el mismo nivel de inteligencia matemática ya que no lo desarrolló durante su carrera, pero tienen más tipos de inteligencia donde nos superan fácilmente.

## **El otro lado**

![Fight Club](https://cdn-images-1.medium.com/max/1600/1*XAxTgtdNQ7BVsFCoI2y4hw.jpeg)

Por curiosidad realicé la misma encuesta a personas técnicas, mismas preguntas pero ahora sobre los comerciales. El resultado, los identificaba como personas poco estructuradas, con ideas poco escalables pero con grandes habilidades sociales.

# No son bugs, son features

![Fix it](https://thepracticaldev.s3.amazonaws.com/i/88x1ygtw5wedjrlbrnbv.gif)

Si todas las personas que contestaron coinciden en los mismos puntos, sólo nos indica que tenemos que trabajar en mejorar nuestros skills sociales, así como los comerciales con la forma de estructuración de ideas.

Considerando ambos lados, en definitiva nos complementamos y podríamos aprender uno del otro para hacer *level up*.
Is anyone faster than Devart? Support for MySQL v8.0 is already in dbForge Studio
Tags: databasetool, mysql, ide, developertool
Published: 2018-06-25 10:20:53

---
title: Is anyone faster than Devart? Support for MySQL v8.0 is already in dbForge Studio
published: true
description: Devart introduced a significant update of dbForge Studio for MySQL with support for MySQL 8.0, MariaDB 10.3, Tencent Cloud. The release also inсludes a brand new feature - navigation between keywords.
tags: database tool, mysql, IDE, developer tool
---
Devart, a Czech software vendor of database tools and data connectivity solutions, has released a significantly improved version of the universal MySQL and MariaDB GUI tool, dbForge Studio for MySQL v8.0. Following the recent update of MySQL Server by Oracle, dbForge Studio now supports for the latest version of MySQL. 

Here’s the scoop on the most outstanding improvements implemented in the updated dbForge Studio for MySQL:

1) With the support for MySQL 8.0, dbForge Studio now offers users the ability to develop, manage and administer MySQL databases on the latest server version using all handy features.

2) dbForge Studio for MySQL continues to broaden the compatibility options and now provides connecting and working with the latest MariaDB v10.3.

3) Good news for Tencent Cloud users—the new dbForge Studio for MySQL v8.0 allows connecting to this cloud service designed for deploying and using MySQL databases in the cloud effortlessly.

4) To make work with code more convenient, dbForge Studio for MySQL v8.0 now offers seamless navigation between keywords in SQL Editor using the shortcut menu or ‘Ctrl+F12’ key combination.

Dive deep into other useful features of dbForge Studio for MySQL v8.0 at https://www.devart.com/dbforge/mysql/studio/ .

About Devart

Devart is one of the leading developers of database tools and administration software, ALM solutions, data providers for various database servers, data integration and backup solutions. The company also implements Web and Mobile development projects.

For additional information about Devart, visit https://www.devart.com/ .
Debugging Tips
Published: 2018-06-25 08:50:50

---
title: Debugging Tips
published: true
description: 
tags: 
---
![](https://cdn.lynda.com/course/112414/112414-636410882080313019-16x9.jpg)

Programmers write code. Programmers aren't perfect. The programmer's code isn't perfect. It therefore doesn't work perfectly the first time. So we have bugs.

We should always employ sound engineering techniques that minimize the likelihood of unpleasant surprises.

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

### Tips

* Reduce bugs to the simplest set of reproduction steps possible.
* Ensure that you are focusing on a single problem.
* Assertions and logging (even the humble console.log) are potent debugging tools. Use them often.
* Binary chop problem spaces to get results faster.
* As you develop your software, invest time to write a suite of unit tests.
* Untested code is a breeding ground for bugs. Tests are your bleach.
* Learn how to use your debugger well. Then use it at the right times.
* Fix bugs as soon as you can. Don't let them pile up until you're stuck in a code cesspit.

**Debugging isn't easy. But it's our own fault. We wrote the bugs.**
Am looking to upgrade my programming laptop, any suggestions less than $600?
Tags: laptop, ask, help, programming
Published: 2018-06-25 08:48:58

---
title: "Am looking to upgrade my programming laptop, any suggestions less than $600?"
published: true
description: 
tags: laptop, ask, help, programming
---


I want a laptop that can work with major IDE's and tools (`android studio, emulators and virtual machines`) smoothly as well as Linux (i have heard HP doesn't have some Linux drivers).
Preferably, `16GB ram` and >= `i5 processor`
> *Lenovo is my preferred brand...*

### What do you suggest or recommenced?
ARM Outputs in VSTS pipelines
Tags: azure, build, cicd, vsts
Published: 2019-09-19 06:42:43

---
title: ARM Outputs in VSTS pipelines
published: true
tags: Azure,Build,CICD,VSTS
canonical_url: https://peterrombouts.nl/2018/06/25/arm-outputs-in-vsts-pipelines/
---

In my work as Cloud Solution Architect I often find teams struggling with the exact same problem. In this blogpost I will highlight one of those problems; passing on ARM output variables within a VSTS pipeline.

Many developers have Powershell or other scripts in place, that fix this problem, but I want to highlight an extension, which is freely available and does this for you, for **free**!

![ARM Outputs by Kees Schollaart](https://keesschollaart.gallerycdn.vsassets.io/extensions/keesschollaart/arm-outputs/3.0.2/1517479176919/images/screenshots-vsts-arm-outputs-1.png)

 

> This extension enables you to use the ARM Deployment outputs in your VSTS environment.
> 
> This step will use the last successful deployment within the selected resource group. If this deployment has outputs, all of them are copied to VSTS variables by the ARM Output key.
> 
> These outputs can then be used by default VSTS ways: `$(same-key-as-in-arm-template)`
> 
> Usually this task is ran directly after the ‘Azure Resource Group Deployment’ task.

So my pro-tip; Stop using your own scripts and implement this handy extension!

PS: On a sidenote, Kees is also a great guy, give him a tweet after you install this in your project ![😉](https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f609.png)

Link to Marketplace: [https://marketplace.visualstudio.com/items?itemName=keesschollaart.arm-outputs](https://marketplace.visualstudio.com/items?itemName=keesschollaart.arm-outputs)  
Kees on Twitter: [https://twitter.com/keesschollaart](https://twitter.com/keesschollaart)
Google photo to Cloudinary
Tags: cloudinary
Published: 2018-06-25 06:15:16

---
title: Google photo to Cloudinary
published: true
description: how to adjust a photo saved in Google photo on Cloudinary
tags: cloudinary
---
Cloudinary is a super cool image adjusting web app.

https://cloudinary.com/

first of all we need to transfer this type of a file, 


to image file, jpeg, png etc..


##How to convert a google photo to a png type photo
you just type jpeg on the end of an URL youve got.


![Alt text of image](http://res.cloudinary.com/db45meith/image/upload/c_scale,e_auto_brightness,w_1301/v1529906850/ojrH_eQc3xMchnYwItR7Jnah7LrEXkH8cVWBJUYrWWBkgsmI0s3xw11MptqSdrIv0y08M143jlboYxjyhmGx5uHuLN5LgzGsXJ3w5UbaKsryj4XF3sWc7NE7LFj-VZ98zixfqFWgSqc_w2400_c3wl5x.jpg)

Google photo < select a photo < share(the URL will be sharten)

it would be help us for this issue.

![converter](https://ctrlq.org/google/photos/)


I made it through [google photo converter](https://ctrlq.org/google/photos/)
then you will get a proper link and upload it to cloudinary. you will enjoy adjusting.
[ann] molten, a modern API framework for Python
Tags: python, web, showdev
Published: 2018-06-25 05:57:35

---
title: [ann] molten, a modern API framework for Python
published: true
description: Announcing my latest project!
tags: #python, #web, #showdev
---

Hey folks,


I just wanted to share this new library I've been working on for a bit now. It's called molten and it's my take on what modern API development with Python should look like. Check it out at https://moltenframework.com and let me know what you think!

If you're interested in the "why" and how it compares with other frameworks, check out the [motivation](https://moltenframework.com/motivation.html) page.
Discuss: Why do developers equate popularity with success for OSS projects?
Tags: discuss, opensource, elm
Published: 2018-06-25 02:09:20

---
title: Discuss: Why do developers equate popularity with success for OSS projects?
published: true
description: I don't buy that adoption is a good way to measure success for a project because the project I contribute to is finding successful according to how we define it. I want to better understand where this association comes from in open source culture.
tags: discuss, open source, elm
---

> Heads up! This post has some claims that lack citations. The power dynamic here is such that I'd rather be dismissed to generally no effect than to put the heat on some developers without a platform who write internet posts that I think are wrong. 

Anyway, let me frame up this discussion a bit:

I am a member of the Elm core team. I made a significant outlay of personal effort to reach this level of contribution, so I think it’s reasonable to claim that I understand what it’s like to care about a project and desire for it to succeed.

A lot of the online lobbying directed at us for changes in prioritization is based on the idea that Elm is not successful unless its community is growing faster now than it was some amount of time ago. This commentary rarely explains how the poster personally defines “success” for Elm, so I have to assume that adoption and success are one and the same in their mind.

For Elm, we reject the idea that adoption is success or even a very good reflection of success. Elm seeks to be an exceptionally good programming language and platform for building front-end web applications. It is successful to the degree that it does that. Major, concrete steps toward this goal will not always result in community growth, especially under Elm's long release cycle. That is okay!

As a final thought, here is an illustrative example of my confusion: if we grant that adoption and success are related, then what does it mean to be the most popular and thus most successful project? If Elm were the most-used front-end technology, what prize would the Elm community win?

So. **What do y'all think is the cause of this apparent linkage in the mind of The Developer?**

I think that if I can better understand why people feel this way then I will be better at explaining how to measure Elm's success to the Elm community.

---

If you would like to share your thoughts anonymously you can email me at hello@lukewestby.com.
Extending Dynamics 365 Package Deployer
Tags: microsoft, dynamics365
Published: 2019-10-09 21:11:48

---
title: Extending Dynamics 365 Package Deployer
published: true
tags: microsoft, dynamics365
canonical_url: https://www.mehmetseckin.com/2018/06/25/extending-dynamics-365-package-deployer.html
---

# Extending Dynamics 365 Package Deployer

25 June 2018, Windsor, UK

The Dynamics 365 Package Deployer is a great tool that allows you to move your customisations and data between environments in bulk.

It imports your solutions and data, and provides an API to hook your custom code up to this process at certain points. These hooks are as follows:

![Custom code hooks for Dynamics 365 Package Deployer process](http://mehmetseckin.com/images/dynamics-365-package-deployer-hooks.png)_Custom code hooks for Dynamics 365 Package Deployer process_

| Method Name | Execution Time |
| --- | --- |
| `InitializeCustomExtension` | After the package deployer extension is initialized, before the solution import process. |
| `PreSolutionImport` | Before importing each solution. |
| `RunSolutionUpgradeMigrationStep` | Before importing each solution, only if the solution that is being imported is already present in the target Dynamics 365 instance. |
| `BeforeImportStage` | Before the solution import completes, if the sample data or flat files are being imported. |
| `AfterPrimaryImport` | After the solution and data imports are completed. |

There are some special parameters and properties that you can set in some of these methods. These parameters and properties can be set to alter the deployment behaviour, such has enabling plug-ins and workflows disabled, overriding safety checks during data import, etc.

There are also some cool methods that lets you interfere with the user interface, such as the `CreateProgressItem`, `RaiseUpdateEvent`, `RaiseFailEvent` methods. These methods enables you to create your own progress items in the progress items pane, however they seem to be only effective after the solution import process begins.

Another cool property that is provided via the Dynamics 365 Package Deployer API is `RootControlDispatcher`. This property allows you invoke logic in the Package Deployer’s main UI thread. This means, you can use your custom WPF user interface during a package deployment.

The Package Deployer also supports customised Welcome and End HTML pages for license agreements, release notes and whatnot. This also supports javascript, however I haven’t tried to run a full-blown single page application on there, and I believe the javascript support would be somewhat limited.

![A JavaScript alert from a custom welcome page in Dynamics 365 deployment package](http://mehmetseckin.com/images/dynamics-365-package-deployer-script-alert.png)_A JavaScript alert from a custom welcome page in Dynamics 365 deployment package_

There is a lot of clever stuff that can be done using these extension points if you want to customise, enhance and automate the deployment process of your Dynamics 365 applications.