teleren/devto huggingface dataset:
How do you debug a Python script that crashes Python itself?
Tags: python
Published: 2023-05-02 17:08:36

---
title: How do you debug a Python script that crashes Python itself?
published: true
description: Sometimes you make Python crash so bad (like segmentation fault) even logs get lost. What do you do?
tags: python
canonical_url: https://www.clpsplug.com/en/2023/05/03/python-debug.html
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-03-29 16:46 +0000
---

## TL;DR

Use `faulthandler.enable()`; Python will print the stack trace to stderr before fully crashing.

## Backstory

Today I tried to daemonize a Python script that calls `request.get` on macOS, and it was insta-dying for no apparent reason.
I tested without daemonization, but the problem only popped up when I daemonize it.

I was doing something like this:

```python
from daemon import DaemonContext
from daemon.pidfile import PIDLockFile
import requests
import time


pid = PIDLockFile(path.join(home_dir, 'pidfile_watcher'))
fp  = open("out.log", "a+")
fpe = open("err.log", "a+")

with DaemonContext(
  pidfile=pid,
  stdout=fp,
  stderr=fpe
):
  while True:
    response = requests.get("https://example.com");
    print(response.text)
    time.sleep(60*10)

fp.close()
fpe.close()
```

Many researches, `print()`s and one final check on Console.app (the macOS standard logger) later, I found that it was segfaulting at requests.get().

## Why did it take so long?

Because daemonized script don't properly log its death even when you pass a file pointer for stderr.  
Also, crash report found in Console.app does not provide much information unless you write most of your program (note the word 'program' - this includes not only your script, but also Python packages and/or even Python itself)! So the only realistic bet may be to spam `print()`, which is already tedious because you need to properly tell `DaemonContext` to redirect stdout to a file (I didn't give any at first, which caused even more headaches).

## The solution

It was `faulthandler.enable()` that I needed to call.

{% embed https://docs.python.org/3/library/faulthandler.html %}

This causes Python to print stack trace to stderr when signal is received (in macOS, segmentation fault causes a signal SIGSEGV; probably all *nix systems also does.) instead of straight-up dying on the spot. This method enabled me to pinpoint exactly which call was causing my script to crash.

## What was the problem, by the way?

Turns out, this is a [known, (probably) unavoidable issue when using request package in macOS.](https://bugs.python.org/issue30385)

To avoid this issue, you must drop proxy support as [segfault occurs at process related to polling OS's proxy configuration, which is "not fork safe".](https://bugs.python.org/issue30385#msg293958)
```python
import os
os.environ['no_proxy'] = "*"
```
Moria Tests
Published: 2023-05-02 16:46:19

---
title: Moria Tests
published: true
description: A brief rant about sanity in testing.
tags: 
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-05-02 16:39 +0000
---
I've recently run across some unit tests that may fail 1) if in the wrong time zone or 2) at the wrong time of day. The latter tests don't immediately appear to have anything to do with time, even.

So I've decided to dub these tests "Moria Tests." This is an allusion to Tolkien's door to Moria that the Fellowship had to arrive at at a precise time in relation to astral phenomena. Any test that is fragile in these ways will henceforth be known thusly by me.

You are hereby granted license to do so as well, without any attribution. And if someone else has already so dubbed them, I apologize and claim ignorance. Must be the universal subconscious at work..
Protecting Your Online Accounts: How to Guard Against Credential Stuffing Attacks.
Tags: digitalidentitymanagement, mfa, adaptiveauthentication, brokenauthentication
Published: 2023-05-02 16:34:28

## **Introduction**

The rise of credential stuffing attacks is a wake-up call for businesses across all industries. In this whitepaper, we'll explore the ins and outs of credential stuffing, including how it works, the types of attacks, and how to defend against them.

Cybercriminals are always on the lookout for new ways to exploit data breaches, and credential stuffing is one of their go-to methods. This whitepaper will provide businesses with valuable insights on how to protect against these attacks.

## **What are Credential Stuffing Attacks?**

In recent years, credential stuffing assaults have become one of the most popular types of cyber intrusions. These attacks function by gaining access to other internet accounts using previously obtained usernames and passwords.

Credential stuffing attacks are particularly effective since many users use the same or similar passwords across several accounts. When one set of credentials is compromised, attackers can exploit it to gain access to other accounts.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/76pd3o17236egehlsz24.png)

## **How to Prevent Credentials Stuffing Attacks?**

In addition to the above measures, businesses can also implement IP blocking, which involves blocking traffic from suspicious IP addresses. This can be effective in stopping credential stuffing attacks that are being carried out by bots using a single IP address. However, attackers often use a large number of IP addresses, making IP blocking a less effective defense mechanism.

## **Conclusion**

In conclusion, **[credential stuffing attacks](https://www.loginradius.com/resource/understanding-credential-stuffing-attacks-whitepaper)** are a significant threat to businesses and their customers. 

To prevent such attacks, businesses must adopt smart technology and security solutions, educate their customers on password hygiene and risks, and have a plan in place for responding to an attack. By taking a proactive approach to cybersecurity, businesses can protect their customers' data and their reputation, ultimately contributing to their long-term success.
Adding authentication to Flask microservice | Software Architecture for beginners, part 4
Tags: beginners, programming, architecture, flask
Published: 2023-05-02 16:25:00

How can you easily add authentication and authorization capabilities in a 3-layered Flask REST API? Welcome to the final lesson of my free "Software Architecture for Beginners" course.


Today we will:

1) Recap 3-layer software architecture pattern
1.1) Benefits of the 3-layer architecture
2) Learn the difference between authentication and authorization
3) Explore how authentication and authorisation logic fits into N-tier architecture
3.1) Where do you put the authentication and authorisation code?
4) Live coding demonstrating - Adding authentication and authorization

{% embed https://youtu.be/eT8D5Mol7kA %}
Survey: Euler: Detecting Network Lateral Movement via Scalable Temporal Link Prediction
Tags: deeplearning
Published: 2023-05-02 16:13:08

## 選定理由

２０２３年当時 PaperwithCode の [Dynamic Link Prediction on Enron Email Dataset](https://paperswithcode.com/sota/dynamic-link-prediction-on-enron-email)で1stだったので選んでみた。

Paper: https://www.ndss-symposium.org/wp-content/uploads/2022-107A-paper.pdf
Paper(ACM): https://dl.acm.org/doi/pdf/10.1145/3588771
Code:
Youtube: https://www.youtube.com/watch?v=1t124vguwJ8

## 概要

【社会課題】
高度に組織化されたサイバー攻撃が増加しており、それに対処するためには、攻撃者の手口や攻撃の痕跡を早期に検知し、適切な対策を講じることが重要である。その中でも、ラテラルムーブメント（[Lateral movement](https://cybersecurity-jp.com/column/33244)）は、サイバー攻撃において重要なステージの一つであるが、検知が困難である。

【技術課題】
ラテラルムーブメントの検知には、膨大なネットワークログを解析し、不審な接続を見つけ出すことが必要となるが、ログは多くの場合非常に膨大でありシステムによって多様である。又、一般的に不審な接続である異常時データは収集が困難である。

【従来技術】
従来の手法では、機械学習を用いた異常検知に注目が集まっており、グラフ理論や深層学習を組み合わせた手法が提案されている。しかし、システムに依存するデータの分布や膨大すぎるデータが原因となり、モデルの汎用性が不足していたり、処理速度・学習時計算量に課題あった。

【提案】
Eulerというフレームワークを提案。このフレームワークは、グラフニューラルネットワークとシーケンスエンコーディングを組み合わせることで、ラテラルムーブメントの検知を行うことができる。また、Eulerモデルは、グラフ畳み込み層を複数のマシンに分散することができるため、処理速度を向上できる。さらに、Eulerモデルは従来の手法と比較して高い検出精度を示しており、実際のシステムで使用することが可能であることを示した。

【効果】
Eulerモデルは、現在の最先端手法と比較して、高精度で大規模なデータ処理にも対応できることを示した。具体的には、従来のモデルに比べてEnronデータセットにて適合率を0.243から0.986の精度でLateral movementを検出できることを示した。また、実世界のネットワークトラフィックに対しても高い精度を発揮し、実用性が高いことが示した。

## 提案：Euler

Eulerは過去の時系列グラフの状態に条件付けられた確率関数を学習し、後の時点でのエッジの発生の可能性を決定する。さらに、単に精度が高いだけでなく、スケーラブルなシステムとして分散型学習の構成の提案も含んでいる。

![fig4](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/al5uy1weyiq1a1mf8js6.png)

Dynamic link detection（動的リンク検出）は、時間的に変化するネットワークデータから、同時に出現するリンク（エッジ）のパターンを検出するタスクであり、具体的には、ある時間ステップで存在しなかったが、あとの時間ステップで存在するようになったリンクを検出することが目的。このタスクは、社会ネットワーク、センサーネットワーク、インターネットトラフィックモニタリングなどの分野で重要である。link detection(リンク予測)は時刻が少し先のリンク検出といえる。
Eulerでは link detectionは帰納的であり、部分観測されたグラフ(G^)からグラフ埋め込み(Z)を経由し、本来のグラフ（G）を再構築する。一方でlink prediction は演繹的であり、推論されたグラフから未来の観測(G^)を予測する。

## 実験

Enron10というデータセットは[このリポジトリ](https://github.com/marlin-codes/HTGN)でも使われている。

![fig3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ywd8bpyvxxv3bfi6ls8h.png)

ベースラインモデル（VGRNN, EGCN）よりもEulerは高精度に検出・予測ができた。Facebookのデータセットで最も改善幅の大きい4%上昇が見られた。このデータセットは求yラフのエッジ数とノード数が最も多く、スナップショットの数が少ない。さ
Routes in React
Tags: react
Published: 2023-05-02 15:52:20

hi guys , i'm new , i started coding in React.js , i have a question , i wrote this "        <Link className="bug-report-button" to="/signalBug" > Signaler un BUG</Link> " , and i also  wrote  "        <Route exact path="/signalBug" component={SignalBug} />        " in App.js my parent component but it does not work ,i want to print a composant when i press a button . 
Thanks for helping me
Try Hack Me - Intro to Docker
Tags: docker, learning, devops
Published: 2023-05-02 15:50:10


Try Hack Me is one of the web sites, where you can learn new skills on interactive way using machines (available 1 in free mode). 

Before this room, also it is available another free room on link: ["Intro to Containerisation"](https://tryhackme.com/room/introtocontainerisation)
One of the new room is "Intro to Docker". Before this room is good to be familiar with Linux Fundamentals, room for this is on the link: [Linux fundamentals](https://tryhackme.com/module/linux-fundamentals). 

In the room you can learn: 

- Basic docker syntax
- Running Your First Container
- Intro to Dockerfiles 
- Intro to Docker compose
- Intro to Docker socket
- Practical 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4aux61oogy214wpm8c2v.png)

Useful post for 50 docker commands you should know and their use cases on link: [50 commands for docker](https://medium.com/@AnnAfame/50-docker-commands-you-should-know-and-their-use-cases-303baff8279d)
Fraud Detection with Graph Databases
Published: 2023-05-02 15:47:29

Graph databases are an effective solution for detecting fraudulent activities in financial transactions, as they can efficiently represent and query complex relationships between entities and transactions. In this post, we'll explore the best practices and examples for detecting fraud with graph databases.

## Data Modeling Best Practices
To model financial transaction data in a graph database, it's important to follow these best practices:

- Identify the key entities: Identify the key entities in the financial transaction network, such as customers, merchants, accounts, and transactions.
- Define node and edge properties: Define node and edge properties to capture important attributes of entities and relationships, such as transaction amount, timestamp, and location.
- Use consistent naming conventions: Use consistent naming conventions for nodes, edges, and properties to make the data model more intuitive and easier to understand.

## Querying Best Practices
Once you've modeled financial transaction data in a graph database, you can query it to detect fraudulent activities. Here are some best practices for querying financial transaction data:

- Use graph algorithms: Use graph algorithms, such as PageRank and community detection, to identify nodes and edges that are likely to be involved in fraudulent activities.
- Use Cypher query language: Use the Cypher query language, which is optimized for graph database querying, to write efficient queries.
- Optimize query performance: Optimize query performance by reducing the number of nodes and edges returned in each query, and by caching frequently accessed data.

## Examples of Fraud Detection Systems
There are several examples of fraud detection systems that have been implemented using graph databases. Here are some popular ones:

- PayPal: PayPal uses a graph database to model customer and merchant relationships, account activity, and transaction history to detect fraudulent activities.
- Mastercard: Mastercard uses a graph database to model cardholder and merchant relationships, transaction patterns, and location data to detect fraudulent activities.
- IBM: IBM uses a graph database to model network activity, user behavior, and security events to detect cyber threats and fraudulent activities.

## Conclusion
In summary, detecting fraudulent activities with graph databases requires careful consideration of data modeling best practices, efficient querying techniques, and the use of specialized algorithms. By following these best practices and using the right algorithms, you can detect fraudulent activities in financial transactions and protect your business from losses.

Check out Apache AGE, an extension for PostgreSQL that lets you build graph databases using SQL and Cypher language on top of relational database.

- [https://age.apache.org/](https://age.apache.org/)
- [https://github.com/apache/age](https://github.com/apache/age)
New redux learner. Help with my code
Tags: redux
Published: 2023-05-02 15:36:44

Hi team, what's wrong with this code? It's doesn't work. I'm new to redux, troubleshooting so far has not fixed it. It's only functionality is to add to an array with increase in number

`import { configureStore } from "@reduxjs/toolkit"
import todoReducer from "../features/todoSlice"

export default configureStore({
    reducer: {
        miniTodo: todoReducer
    }
})`

`import { useSelector, useDispatch } from 'react-redux'
import { addToArray } from './features/todoSlice'

export function TodoList () {
    const count = useSelector(state => state.miniTodo.value)
    const dispatch = useDispatch()

    return (
        <div>
            <button onClick={()=> dispatch(addToArray())}>Add Item</button>
        </div>
    )
}
`
`import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import reportWebVitals from './reportWebVitals';
import store from './app/store';
import { Provider } from 'react-redux';
import { TodoList } from './todo';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <Provider store={store}>
    <TodoList />
  </Provider>
);`

`import { createSlice } from "@reduxjs/toolkit"

export const todoSlice = createSlice({
    name: "miniTodo",
    initialState: {
        value: []
    },
    reducers: {
        addToArray: state => {
          const newThingText = `Thing ${state.value.length + 1}`
          state.value.push(newThingText)
          console.log(state.value)
        },
        useNotKnown: (state, action) => {
          state.value += action.payload
        }
    }
})

export const { addToArray, useNotKnown} = todoSlice.actions

export default todoSlice.reducer
`
Diagram editor dgrm.net / Announcement / Stroke
Published: 2023-05-02 15:30:58


![Dependency diagram editor. Stroke for groups of shapes.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xosrbaudiuqoh8iuasn1.png)

You can now stroke groups of shapes.
An example of a chart with a stroke:
[https://app.dgrm.net/?k=202305e2c36abd](https://app.dgrm.net/?k=202305e2c36abd)

Announcements on Twitter: https://twitter.com/boyko_tech
Анонсы в Telegram: https://t.me/dgrm_net
{#2}Variables in JavaScript | JavaScript
Published: 2023-05-02 15:29:44

console.log("This repl contains code for the  JavaScript ")
var a = 67 // a contains 67
console.log(a)
a = "Sage"
console.log(a)
let 8Sage = 7 // Not allowed this will throw an error
// let var = 7[](https://replit.com/@SkSage/02Variblesdatatype)
Changing careers
Tags: beginners, javascript, css, html
Published: 2023-05-02 15:17:24


![Photo by <a href="https://unsplash.com/@soymeraki?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Javier Allegue Barros</a> on <a href="https://unsplash.com/photos/0nOP5iHVaZ8?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
  ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bkn53exbel0um72k2a1a.jpg)**Changing careers** is exciting and intimidating, and some people think that starting a new job is challenging and impossible for adults and seniors. I was a geologist long ago, but I am changing my major from geology to software development. There are two reasons why I am changing careers: first, my long-term goals have changed, and second, I realized that dealing with software and data is my passion.

First, I decided to change careers because my long-term goals have changed. I was born in Venezuela, where I got my degree as a geologist and started a family. However, Venezuela's political and economic crisis is worsening, so I decided to create a new life abroad. I began to work at Amazon in the United States in 2020, and I have enrolled in the Amazon Career Program, which allows employees to work and study part-time careers. Thanks to the Amazon Career Program, I have gotten my GED and language certificate, and next October, I will get my certification as a Full Stack Developer.

Second, after several years of working with databases and geostatistical modeling software, I realized that my current passion is working with programming languages, software, and websites. As a result, I decided to pursue a software development career.

In brief, I'm changing careers because my long-term goals have changed, and I realized that my passion is working with programming languages, software, and websites. I have studied at Flatiron School for about a month and learned basic HTML, CSS, and JavaScript concepts and methods. I'll try my best.
 
Ivan
WPCS: Use short array syntax
Tags: wordpress, phpcs
Published: 2023-05-02 15:14:28

At Plank, we use the [WordPress Coding Standards](https://developer.wordpress.org/coding-standards/wordpress-coding-standards/php/) in our development process. It helps ensure that everyone is writing code in a similar style, and that we follow best practices (like [escaping](https://developer.wordpress.org/apis/security/escaping/) and [sanitizing](https://developer.wordpress.org/apis/security/sanitizing/) data).

That said, we don't necessarily agree with all of the rules. In particular, we prefer using the [short array syntax](https://make.wordpress.org/core/2019/07/12/php-coding-standards-changes/) for php arrays. We find it easier to find when an array closes, rather than searching through a sea of closing parentheses (`);););`).

If you're using your own custom ruleset to extend the default standards, you do something like this to allow the short array syntax. If you _really_ don't like the long array syntax, you can disallow it, and `phpcbf` should convert to short arrays with the second pattern.

```xml
<!-- Allow short syntax arrays. -->
<rule ref="Generic.Arrays.DisallowLongArraySyntax"/>

<!-- Disallow long array syntax -->
<rule ref="Generic.Arrays.DisallowShortArraySyntax.Found">
	<exclude-pattern>*</exclude-pattern>
</rule>
```
The advantages of using the MEAN stack for full-stack web development projects
Tags: hire, mean, developers, programming
Published: 2023-05-02 15:14:08

The MEAN stack is a popular choice for full-stack web development projects, offering a range of advantages that make it an attractive option for businesses and developers alike. MEAN stands for MongoDB, Express.js, AngularJS, and Node.js, and each of these technologies brings unique benefits to the table.

One of the primary advantages of the MEAN stack is its versatility. It is a full-stack solution that can handle everything from front-end development to back-end programming, making it an all-in-one solution that saves time and resources. It also offers the flexibility to build and deploy applications quickly and easily, as well as the ability to scale up or down as needed.

Another advantage of the MEAN stack is its open-source nature. This means that the technologies are constantly evolving and improving, with a large community of developers contributing to their development. This translates to faster and more efficient development processes, as well as more reliable and secure applications.

Finally, the MEAN stack is known for its performance, as it is built on powerful and efficient technologies that can handle large-scale applications and heavy traffic. This makes it an ideal choice for businesses that need to build robust and scalable web applications.

To take advantage of the benefits of the MEAN stack, businesses can [hire MEAN developers](https://www.aspired.io/blog/how-hire-mean-developers-2023-beginners-guide) who are experienced in using these technologies to build high-quality web applications. With the right team in place, businesses can streamline their development processes, save time and resources, and create applications that are both reliable and efficient.
View your Development Site on your own Devices!
Published: 2023-05-02 15:09:59

Developer tools in most modern browsers to allow us to view our site on different device sizes, but did you know you can also view your site while developing on an actual device? Read on and I'll share how you can view your site on your actual devices whilst in development. 

*Note:* I've currently tested this and it works for static, Flask and Django projects. Let me know what you try it with in the comments below 😊


### Steps

1. Make sure that your computer and mobile device are both connected to the same wifi network.

2. Find your local IP address:

    **On Windows:**

    1. Open the terminal and run `ipconfig`

    2. This will return your IP configuration. Find and make a note of the IP4 address value.

    
    **On Mac:**

    1. Open the terminal and type `ipconfig getifaddr en0`

    2. This will return the IP4 address value. Make a note of it.

    
3. Once you have the local IP address, run your project. You may do this by using the Live Server extension in VSCode or by running the server command, for example python3 manage.py runserver for Django or python3 app.py for Flask applications. Make a note of the port your project is running on.

4. On your mobile device, open a browser window and enter the IP address followed by a colon and the port number. Voila! you should now be able to see your project 😊

As you are connected to the same IP address and port that your computer is using, you can make changes and view them in real time simply by refreshing your browser.

Its always good to know several ways to achieve the same thing in web development, and this may help if you ever run into an odd bug that only displays on the device, but not in dev tools (we've all been there! 🪳)