teleren/devto huggingface dataset:
How To Monitor Http traffic Of Third Party Service
Tags: monitoring, tutorial, linux
Published: 2022-09-04 08:54:02

## Why

The company purchased a third-party service and deployed it on the company's machine in the form of a jar package.This service is on a critical data link, and the concurrency is very high. The service http request appears 500, which is what we cannot receive.

Even without the source code, we also need to monitor HTTP requests to provide detailed information to the provider.

## Ideas to Solve the Problem

My first thought was to use java agent to monitor.Java agents are a special type of class which, by using the Java Instrumentation API, can intercept applications running on the JVM, modifying their bytecode.

Another solution is to capture network traffic. My core requirement is to monitor requests with a status code of 5xx, so capturing traffic is more suitable.

## Capture network traffic

Tcpdump can capture traffic but needs parsing to be more suitable for reading.

httpry is a tool designed for displaying and logging HTTP traffic.

Nice!This is the tool I was looking for.

### Install httpry on Centos

```jsx
sudo yum install epel-release

sudo yum install httpry
```

### **Filter Response with a Status Code other than 200**

```jsx
sudo httpry -m post -f timestamp,source-ip,dest-ip,direction,status-code -o test.log 'tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2)+8:4] != 0x20323030'
```

The output in test.log like this:

```jsx
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       403
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       403
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
```

### Manager Large Data Volume Output File

All the information is typed into one file, this file will be very large.

You need another tool:logrotate.

I want to organize the files by date and I create the file logrotate.conf

```jsx
/xxxx/xxxxx/http.log {
    notifempty
		copytruncate
		dateext
		dateformat .%Y%m%d
		olddir /xxx/backup
}
```

The following command can debug the process
```
logrotate -d xxx/lograte.conf
```
Then add timed tasks through ctontab，crontab will execute the rotate according to the config and We will get a file named http.log.20220815.

## More Resources
[httpry](https://github.com/jbittel/httpry)
[logrotate](https://github.com/logrotate/logrotate)
Webinar! Become a Tabnine expert in 40 minutes ⚡️
Tags: webinar, webdev, programming, productivity
Published: 2022-09-04 08:53:09


Join our webinar tomorrow and learn all about:

🎯 Which plan best suits you?
🎯 Foundational features of Tabnine
🎯 The Tabnine Hub
🎯 How to get the best results

⏰ Monday | September 5th | 5PM CET

{% embed https://youtu.be/XXERCwezdsQ %}.
Importing SASS Partials Breakdown
Tags: productivity, programming, webdev, tutorial
Published: 2022-09-04 08:41:11

Partials in SASS help us to break up our files into smaller files without affecting performance.

The use of partials allows us to modularize our CSS, to help keep things maintainable.

We divide up our Sass into separate files representing different components. A partials’ name will always start with an underscore _. We then import the partial using an `@import` directive.

## Using Partials
Let’s say our Sass file is getting rather large. We might choose to make a partial file that contains just the code that’s relevant to the header section; we’d call it `_header.scss` and move the appropriate code into that new file.

We would then import it back into `main.css`, like so:
```css
// In main.scss
@import 'header';
```
_Note_: When you import a file, there’s no need to include the _ or .scss file extension.

## @import in CSS vs SASS
The `@import` directive is of course, also available in CSS. However, there is an important difference. Whenever an import statement is used in CSS, an HTTP request is made to the server. This increases the amount of resource requests and negatively affects the performance of a web page. SASS does not do that. SASS takes the file that you want to import from and combines it with the file you’re importing. So you can serve a single CSS file to the web browser, which does not affect the performance.

In the next post in this series, we’ll be looking at how to implement inheritance with `extends`.

## Conclusion
If you liked this blog post, follow me on [Twitter](https://twitter.com/RembertDesigns) where I post daily about Tech related things!
![Buy Me A Coffee](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ijinng8k7bf23v0o6gkj.png) If you enjoyed this article & would like to leave a tip — click [here](https://www.buymeacoffee.com/rembertdesigns)

### 🌎 Let's Connect

- [Portfolio](https://www.rembertdesigns.co/)

- [Twitter](https://twitter.com/RembertDesigns)

- [LinkedIn](https://www.linkedin.com/in/rrembert/)

- [Hashnode](https://rembertdesigns.hashnode.dev/)

- [Devto](https://dev.to/rembertdesigns)

- [Medium](https://medium.com/@rembertdesigns)

- [Github](https://github.com/rembertdesigns)

- [Codepen](https://codepen.io/rembertdesigns)
Closures #React Qucik Notes.
Tags: react, javascript, node, beginners
Published: 2022-09-04 08:19:42

**Why we need to Understand closure?**

> So we can know where to declare variable and which function in nested level can have access to it.

For understanding Closure, please refer below link which has provided best documentation to understand.

_https://developer.mozilla.org/en-US/docs/Web/JavaScript/Closures_
redis-持久化
Published: 2022-09-04 08:19:41

## 一、RDB快照持久化

1. 
在指定时间间隔将内存数据写入磁盘，默认持久化方式

1. 
触发方式

```
1. 执行save和bgsave命令
2. 配置文件设置save <seconds> <changes>规则，自动间隔性执行bgsave命令
3. 主从复制时，从库全量复制同步主库数据，主库会执行bgsave
4. 执行flushall命令清空服务器数据
5. 执行shutdown命令关闭Redis时，会执行save命令
```

1. 
save命令
save会阻塞redis进程，再快照创建完之前不能出来其他请求

1. 
bgsave命令

- 
bgsave不会阻塞redis进程，会fork出一个进程把全量数据写入临时rdb文件，再替换正式rdb文件

- 
配置文件自动触发bgsave
```
# 900秒内有1次key改变则触发
save 900 1
# 300秒内有10次key改变则触发
save 300 10
# 60秒内有10000次key改变则触发
save 60 10000
```

1. 
优点：

- 
RDB快照是一个压缩过的非常紧凑的文件，保存着某个时间点的数据集，适合做数据的备份，灾难恢复

- 
可以最大化Redis的性能，在保存RDB文件，服务器进程只需fork一个子进程来完成RDB文件的创建，父进程不需要做IO操作

- 
与AOF相比，恢复大数据集的时候会更快

1. 
缺点：

- 
RDB的数据安全性是不如AOF的，保存整个数据集的过程是比繁重的，根据配置可能要几分钟才快照一次，如果服务器宕机，那么就可能丢失几分钟的数据

- 
Redis数据集较大时，fork的子进程要完成快照会比较耗CPU、耗时


## 二、AOF持久化
```
# appendonly参数开启AOF持久化
appendonly no

# AOF持久化的文件名，默认是appendonly.aof
appendfilename "appendonly.aof"

# AOF文件的保存位置和RDB文件的位置相同，都是通过dir参数设置的
dir ./

# 同步策略(always/everysec/no)
appendfsync everysec

# aof重写期间是否同步
no-appendfsync-on-rewrite no

# 重写触发配置
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# 加载aof出错如何处理
aof-load-truncated yes

# 文件重写策略
aof-rewrite-incremental-fsync yes

# 同时开启rdb、aof
aof-use-rdb-preamble yes
```

1. 
append命令

- 
开启aof，每个执行一个命令，都会把该命令以协议格式先追加到aof_buf缓存区的末尾，而不是直接写入文件，避免每次有命令都直接写入硬盘，减少硬盘IO次数

- 
aof_buf写入aof策略：
```
appendfsync always：
将aof_buf缓冲区的所有内容写入并同步到AOF文件，每个写命令同步写入磁盘

appendfsync everysec：
将aof_buf缓存区的内容写入AOF文件，每秒同步一次，该操作由一个线程专门负责

appendfsync no：
将aof_buf缓存区的内容写入AOF文件，什么时候同步由操作系统来决定
```

1. 
rewrite命令

- 
随着时间推移，aof文件会越来越大，rewrite就是为了缩小aof文件的大小

- 
rewrite不是读取aof文件，而是读redis生成新的aof文件替换老的aof文件

- 
redis-7会把数据保存到aof目录下的rdb中，同时清空aof文件

- 
命令触发
```
# fork出一个进程来执行，服务器继续执行命令，且最加到aof_buf、
# aof_rewrite_buf
127.0.0.1:6379> bgrewriteaof
```

- 
自动触发
配置文件：
```
# 当AOF文件的体积大于64MB，且AOF文件的体积比上一次重写后的体积大了一倍（100%）时，会执行`bgrewriteaof`命令
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
```

1. 
优点：

- 
数据更完整，安全性更高，秒级数据丢失（取决fsync策略，如果是everysec，最多丢失1秒的数据）

- 
AOF文件是一个只进行追加的日志文件，且写入操作是以Redis协议的格式保存的，内容是可读的，适合误删紧急恢复

1. 
缺点：

- 
对于相同的数据集，AOF文件的体积要大于RDB文件，数据恢复也会比较慢

- 
根据所使用的 fsync 策略，AOF 的速度可能会慢于 RDB。 不过在一般情况下， 每秒 fsync 的性能依然非常高
python_requests_05肯德基餐厅位置查询器
Published: 2022-09-04 08:08:45

```
#! C:\Users\neo\Desktop\pyfiles\venv python
# -*- coding:utf-8 -*-
import requests
import json
if __name__ == "__main__":
    #指定url
    post_url = 'http://www.kfc.com.cn/kfccda/ashx/GetStoreList.ashx?op=keyword'
    #UA伪装
    headers = {
        'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36'
    }
    #post请求参数处理
    word = input('Enter a China city name:')
    data = {
        'cname':'',
        'pid':'',
        'keyword': word,
        'pageIndex':'1',
        'pageSize': '10',
    }
    #请求发送
    response = requests.post(url=post_url,data=data,headers=headers)
    #获取响应数据
    page_text = response.text
    #持久化存储
    fileName = word+'肯德基位置查询.txt'
    fp = open(fileName,'w',encoding='utf-8')
    json.dump(page_text,fp=fp,ensure_ascii=False)
    print('成功结束！')
```
python_requests_04豆瓣电影排行榜爬取
Published: 2022-09-04 08:07:15

```
#! C:\Users\neo\Desktop\pyfiles\venv python
# -*- coding:utf-8 -*-
import requests
import json
if __name__ == "__main__":
    #指定url
    url = 'https://movie.douban.com/j/chart/top_list?'
    param = {
        'type': '11',
        'interval_id': '100:90',
        'action':'',
        'start': '0',    #从库中的第几部电影开始爬取
        'limit': '30',   #单次爬取的个数
    }
    headers = {
       'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36'
   }
    response = requests.get(url=url,params=param,headers=headers)
    #获取响应数据  json()方法返回的是obj(需确认响应数据为json格式)
    list_data = response.json()
    #持久化存储
    fp = open('豆瓣剧情排行榜.json','w',encoding='utf-8')
    json.dump(list_data,fp=fp,ensure_ascii=False)

    print('成功结束！')
```
python_requests_03百度翻译自动查询器
Published: 2022-09-04 08:03:35

```
#! C:\Users\neo\Desktop\pyfiles\venv python
# -*- coding:utf-8 -*-
import requests
import json
if __name__ == "__main__":
    #指定url
    post_url = 'https://fanyi.baidu.com/sug'
    #UA伪装
    headers = {
        'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36'
    }
    #post请求参数处理
    word = input('Enter a word:')
    data = {
        'kw': word,
    }
    #发送post请求
    response = requests.post(url=post_url,data=data,headers=headers)
    #获取响应数据并处理    json()方法返回的是obj(需确认响应数据为json格式)
    dict_obj = response.json()
    #持久化存储
    fileName = word+'.json'
    fp = open(fileName,'w',encoding='utf-8')
    json.dump(dict_obj,fp=fp,ensure_ascii=False)

    print('成功结束！')
```
python_requests_02搜索结果页面采集器
Published: 2022-09-04 08:01:58

```
#! C:\Users\neo\Desktop\pyfiles\venv python
# -*- coding:utf-8 -*-
import requests
if __name__ == "__main__":
    #UA伪装：将对应的User-Agent封装到一个字典中。
    headers = {
        'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36'
    }
    url = 'https://www.sogou.com/web'
    #处理url携带的参数：封装到字典中
    kw = input('Enter a word:')
    param = {
        'query':kw
    }
    #对指定的url发起的请求携带了一些参数。
    response = requests.get(url=url,params=param,headers=headers)

    page_text = response.text
    fileName = kw+'.html'
    with open(fileName,'w',encoding='utf-8') as fp:
        fp.write(page_text)
    print(fileName,'保存成功！')
```
Feature Flags On Rails Routes
Tags: rails, flipper
Published: 2022-09-05 01:47:20

---
title: Feature Flags On Rails Routes
published: true
date: 2022-09-04 08:01:01 UTC
tags: rails,flipper
canonical_url: 
---

Found a [tweet](https://twitter.com/julian_rubisch/status/1565244598204645376) explaining how to wrap a rails route with a feature flag.

```
resources :media,
      constraints: ->(req) do
                     current_user = req.env["warden"].user(:user)
                     Flipper.enabled?(:media_uploads, current_user)
                   end

```

Source: https://twitter.com/carbon\_noreply/status/1565244390666252288
python_requests_01爬取搜狗首页
Published: 2022-09-04 07:55:21

```
#! C:\Users\neo\Desktop\pyfiles\venv python
# -*- coding:utf-8 -*-
#需求：爬取sogou.com首页
import requests
if __name__ == "__main__":
    #指定url
    url = 'https://www.sogou.com/'
    #发起请求        #get方法会返回一个响应对象
    response = requests.get(url=url)
    #获取响应数据     #.text返回字符串形式的响应数据
    page_text = response.text
    print(page_text)
    #持久化存储
    with open(r'C:\Users\neo\Desktop\pyfiles\sogou.html','w',encoding='utf-8') as fp:
        fp.write(page_text)
    print('爬取数据结束！')
```
Centos7上搭建DVWA网站（基于宝塔）
Published: 2022-09-04 07:39:43

1、安装宝塔套件——宝塔官网有远程安装代码
https://www.bt.cn/bbs/thread-19376-1-1.html
2、下载DVWA并上传至/www/wwwroot目录
3、修改config.inc.php中的数据库链接用户密码——与宝塔面板中一致
4、访问宝塔面板并新建网站、新建数据库
5、实体机域名解析——hosts文件中添加192.168.8.132  dvwa.org
6、浏览器访问dvwa.org并点击网页底部的”Create/Reset Database"，自动跳转登录页面后admin/password访问。
Coming soon page using JavaScript
Tags: javascript, webdev, css, html
Published: 2022-09-04 06:38:59

Today we will share with you a snippet about Coming Soon or Maintenance Page Design. What is a coming soon landing page? Suppose you have a website. You want to add a new feature or product to your website. The layout of the website will be chaotic because you will be messing with the coding, which is not what you want your visitors to see. In this situation, you can create this coming soon landing page and show it to the visitors. This means that when someone searches your web URL, they will see this page. They will understand that website is under maintenance. Usually, a coming soon or maintenance page has a countdown clock. along with a headline text which indicates what the page is all about. it also contains the company logo, a brief text about why the maintenance is going on, and a call to action button. Coming soon landing pages are usually very simple. **[Download Code](https://www.divinectorweb.com/2022/09/coming-soon-page-using-javascript.html)**
Promise in Javascript #Node Quick Notes.
Tags: javascript, react, node, beginners
Published: 2022-09-04 07:30:52

**Take it easy as a javascript if else statement that we use every time whether a condition is true or not.**

`If(we got success in something){
return resolution of successs;
}else{
return rejection;
}`

As above same a Promise can be defined but remember to use new key word while defining it.

```javascript
const myPromise = new Promise(
(success,failure) => { if(true){success("resolve");} else {failure("reject");};}
);

const functionToExecutePromise = () => {
  myPromise
    .then(ok => {
      console.log(ok)
    })
    .catch(err => {
      console.error(err)
    })
}

functionToExecutePromise();

```
Boilerplate for Node-Express with sequelize ORM
Tags: javascript, node, boilerplate, express
Published: 2022-09-04 07:12:39

https://github.com/kazi-naimul/node-express-mysql-boilerplate

A boilerplate for any enterprise rest api or service with Node.js, Express and Sequelize ORM for mysql, postgresql or others.

By running this project you will get a production ready environment with all necessary supports for validation, unit testing, socket, redis and many more.
Javascript Timers #React Qucik Notes.
Tags: javascript, reac, node, nodequciknotes
Published: 2022-09-04 07:10:34

**Remeber below functions from which we can do all timer related things.**

**1. SetTimeout();**

```javascript
const myFunc = () => {return setTimeout(() => {console.log("executed after 2 sec")},2000);};

myFunc();
```

**2. setInterval();**

```javascript
const myFunc = () => {return setInterval(() => {console.log("executing every 2 sec")},2000);};

myFunc();

//ClearInterval(myFunc); //Call this function to clear running interval
```
With Laravel Package you can create instance real-time validation for your input in blade.
Tags: laravel, blade, realtime, validation
Published: 2022-09-04 07:10:33

## Installation

Install the package by using composer:

```bat
composer require yemeni-open-source/blade-realtime-input
```

## Basic Usage

The `<input>` tag:

```php
<x-realtime-input::strings name="username" rules="required|min:5" />
```

The `<select>` tag:

```php
<x-realtime-input::options 
    rules="in:usd,yer"
    class="btn btn-default custom-select"
    name="currency" id="currency"
    >
    <option value="usd">USD</option>
    <option value="yer" selected>YER</option>
    <option value="sar">SAR</option>
</x-realtime-input::options>
```

## Advance Usage

You can add `id`, `class` , `type` or other HTML attributes to your input like following

```php
<x-realtime-input::strings
    rules="required|min:5"
    name="username"
    id="user"
    class="form-control"
    />
```

You can add array name like following

```php
<x-realtime-input::strings
    name="username[]"
    rules="required|min:5"
    id="user"
    class="form-control"
    />
```

## Localization

The library works as you familiar with laravel validator you can read laravel validation localization.
mmm, I am devastated
Published: 2022-09-04 07:08:39

Hello Lads, I am here today to tell about my life ( I think so ). There have been many things that happened to me.

I was not active because I had school and a lot to study. Also the exams were near. Now, we have 10 days leave ( From 2 September to 12 September ) due to Onam ( A festival celebrated in India. I am free now as nothing to learn and study. So I thought about uploading a YouTube video as i was not active in YouTube also. 

There comes the _problem_; I got cough and fever and my health was so bad, I could not even stand up due to the effects of the medicine. Now I am a little better and then i thought - Why not to make a video now or put up a post in Dev Community?
Then comes the next _problem_; I don't have any ideas! I am thinking day and night and I am requesting to the community to give me an idea. 

My videos don't have my voice over. It is just the video, keyboard clicking sound, and a bubble to say the intro ( as devaslifes videos ). 

Now, this is what made me devastated. I can't do nothing and hope is my last chance. I want to keep my channel active, study, being social ( talking with humans ) etc. I don't know if I can say devastated but my brain is. 

Now I getting back. On 7th is my Birthday; so before that I need to put up a post in Dev and post a video in YouTube before the vacation ends. That ends my devastating things. If you have any problem in this post, like devastated word is no good etc. please comment.

Signing Up,
Gautham.
What is the best way to develop a offline first application in React Native?
Tags: discuss, reactnative
Published: 2022-09-04 07:07:17

Offline applications is a feature that permits the user to access mobile apps without the Internet.

On my requirement, I would like to create an app that allows users to save data such as pictures and videos while they are offline and upload them when they connect to the internet.

What is the best way to achieve my goal, and what are my data storage options?
Track Anyone's IP Address using Python
Tags: python, opensource, api, programming
Published: 2022-09-04 06:53:45

Using this technique, you can make your own IP Tracker.

At first, download python in your system.

**Step 1 :** Add Dependencies

```python
import os
import urllib2
import json
```
**OS Module :** It will be used to interact with Operating System.
<br>
**urllib2 Module :** It will be used to handle "urls".
<br>
**json(JavaScript Object Notation) Module :** It will be used to store and exchange data.
<br>

**Step 2 :** Take Raw Input From User (Tageted IP)

```python
ip = input("What is your target IP: ")
```
**Step 3 :** Enter URL, whose api we are using

```python
url = "http://ip-api.com/json/"
```
**Step 4 :** Get HttpResponse in a variable

```python
response=urllib2.urlopen(url+ip)
```
**Step 5 :** Get Data from response in a variable

```python
data=response.read()
```
**Step 6 :** Get Data in JSON form in a variable

```python
values=json.loads(data)
```
**Step 7 :** Showcase the Informations

```python
print("IP: "+values["query"])
print("City: "+values["city"])
print("ISP: "+values["isp"])
print("Country: "+values["country"])
print("Region: "+values["region"])
print("Timezone: "+values["timezone"])
```
Full Code : [Link](https://github.com/CatalystsReachOut/python/blob/master/PythonScripts/MakeIPTracker/main.py)
Great pieces of advice for programmers i came across on the internet.
Tags: beginners, career, programming, productivity
Published: 2022-09-04 06:42:01

## Don’t call yourself a programmer👨‍💻

"Programmer" sounds like "anomalously high-cost peon who types some mumbo-jumbo".Instead, describe yourself by what you have accomplished for employers or from your projects - increasing revenues/reducing costs/implementing efficient systems.

## Mix Mediums

Some people learn best by reading, others by watching videos, but if you mix it up — watch a video, then do some reading, then practice with some interactive code sessions, you’ll repeat the concepts from multiple angles, and multiple examples.

## Share, Document 📝

As a developer, learning how to code is only part of t the equation - you also need to be able to articulate what you're doing and why. 

Practice explaining, documenting, and teaching the concepts to your coworkers and other collaborators on your projects.

When you're working with other developers, your code will be reviewed by other people, and you may find that you didn't understand well enough to defend your choices.

## Reading code

Reading code can help you develop a code sense that will alert you when you make a mistake, help you spot others' mistakes, and code smell. Initially, you will find it boring and difficult, but with time you will develop a good code sense.

Code sense is one of the signs of a better programmer, they often tend to look, at what you missed.

## Accept failures 

Being open to failure is crucial to encouraging patience and learning.

Making mistakes and failing can be the most important experiences in the learning process. Allow for that instead of creating an environment where it isn't safe to take risks or grow.

## Understand what you are doing

when someone tells you about something, such as a design pattern, coding practice, algorithm, seek to understand its limitations and advantages. Use that understanding to make wise decisions about using it and what modifications can make it even more appropriate to your application.
Very usefull Backtick(`) or Template literal #React Qucik Notes.
Published: 2022-09-04 06:34:10

**Template Literral or Backtick is for Javascript string combination. **
For an example,

```javascript
const myFunc = (parameter1, parameter2) => {return `This will result in combination of this paramter 1 value = ${parameter1}  and parameter2 value = ${parameter2}  when executed in console.`}

myFunc(1,2);
```
**Remember below things while using Template Literral.**

**To escap backtick in Template literral use backslash()** for e.g.

```javascript
const myFunc = () => {return `This is an example of back tick include in this string here next to me \` and it will work.`}

myFunc();
```