title: Ð­ÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾Ðµ Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ðµ Ð¼Ð¾ÑÐµÐ½Ð½Ð¸ÑÐµÑÐºÐ¸Ñ ÑÑÑÐ»Ð¾Ðº Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ Ð¼Ð°ÑÐºÐ¸ÑÑÑÑÐµÐ³Ð¾ÑÑ Ð¿Ð¾Ð´ Ð¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°ÑÐµÐ»Ñ Ð²ÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑÐ°
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¼Ð°Ð³Ð¸ÑÑÑÐ°
titleType: Subtitle
title: Effective detection of fraudulent links using a disguised web scraper
Creators: name: ÐÐ°ÑÐ°ÑÐ¾Ð², ÐÐ»ÐµÐºÑÐ°Ð½Ð´Ñ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2024
Subjects: ÑÐ°Ð·Ð´ÐµÐ»ÐµÐ½Ð¸Ðµ ÑÑÐ°ÑÐ¸ÐºÐ°
Ð²ÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑ
Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ðµ Ð¼Ð¾ÑÐµÐ½Ð½Ð¸ÐºÐ¾Ð²
Ð¼Ð°ÑÐºÐ¸ÑÐ¾Ð²ÐºÐ° ÑÐºÑÐ°Ð¿ÐµÑÐ°
Ð²ÑÐ±Ð¾ÑÐ¾ÑÐ½Ð°Ñ Ð·Ð°Ð³ÑÑÐ·ÐºÐ° ÑÐµÑÑÑÑÐ¾Ð²
cloaking
web scraper
fraud detection
scraper disguise
selective resource loading

description: Ð Ð°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² ÑÐ°Ð·Ð´ÐµÐ»ÐµÐ½Ð¸Ñ ÑÑÐ°ÑÐ¸ÐºÐ° Ð½Ð° Ð¼Ð¾ÑÐµÐ½Ð½Ð¸ÑÐµÑÐºÐ¸Ñ ÑÐ°Ð¹ÑÐ°Ñ Ð¸ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐµ Ð¼ÐµÑÐ¾Ð´Ð¸ÐºÐ¸ Ð´Ð»Ñ Ð¸Ñ Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ Ð²ÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑÐ°, Ð¼Ð°ÑÐºÐ¸ÑÑÑÑÐµÐ³Ð¾ÑÑ Ð¿Ð¾Ð´ Ð¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°ÑÐµÐ»Ñ. ÐÐ°Ð´Ð°ÑÐ¸ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ: 1. ÐÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¸Ðµ Ð¾Ð±Ð·Ð¾ÑÐ° Ð¸Ð½ÑÑÑÑÐ¼ÐµÐ½ÑÐ¾Ð² Ð´Ð»Ñ Ð¿Ð¾ÑÑÑÐ¾ÐµÐ½Ð¸Ñ Ð²ÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑÐ° Ð² ÐºÐ¾Ð½ÑÐµÐºÑÑÐµ Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ ÑÐ°Ð¹ÑÐ¾Ð² Ñ ÑÐ°Ð·Ð´ÐµÐ»ÐµÐ½Ð¸ÐµÐ¼ ÑÑÐ°ÑÐ¸ÐºÐ°; 2. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð¾Ð² Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ ÑÐ°Ð¹ÑÐ¾Ð² Ñ ÑÐ°Ð·Ð´ÐµÐ»ÐµÐ½Ð¸ÐµÐ¼ ÑÑÐ°ÑÐ¸ÐºÐ°; 3. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð²ÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑÐ° Ð´Ð»Ñ Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ ÑÐ°Ð¹ÑÐ¾Ð² Ñ ÑÐ°Ð·Ð´ÐµÐ»ÐµÐ½Ð¸ÐµÐ¼ ÑÑÐ°ÑÐ¸ÐºÐ°; 4. Ð¢ÐµÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð½Ð¾Ð³Ð¾ ÑÐµÑÐ²Ð¸ÑÐ° Ð¸ Ð¾ÑÐµÐ½ÐºÐ° ÐµÐ³Ð¾ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸. ÐÐ»Ñ Ð¼Ð°ÑÐºÐ¸ÑÐ¾Ð²ÐºÐ¸ Ð²ÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑÐ° Ð¸ÑÐ¿Ð¾Ð»ÑÐ·ÑÐµÑÑÑ ÑÐ¾ÑÐ°ÑÐ¸Ñ IP-Ð°Ð´ÑÐµÑÐ¾Ð², Ð¿Ð¾Ð´Ð¼ÐµÐ½Ð° HTTP-Ð·Ð°Ð³Ð¾Ð»Ð¾Ð²ÐºÐ¾Ð², ÑÐ¼ÑÐ»ÑÑÐ¸Ñ ÑÐ°ÑÐ°ÐºÑÐµÑÐ¸ÑÑÐ¸Ðº ÑÑÑÑÐ¾Ð¹ÑÑÐ²Ð° ÑÐµÑÐµÐ· Ð½Ð°ÑÑÑÐ¾Ð¹ÐºÑ JavaScript Ð¾ÐºÑÑÐ¶ÐµÐ½Ð¸Ñ Ð±ÑÐ°ÑÐ·ÐµÑÐ°. ÐÐ±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ðµ ÑÐ°Ð¹ÑÐ¾Ð² Ñ ÑÐ°Ð·Ð´ÐµÐ»ÐµÐ½Ð¸ÐµÐ¼ ÑÑÐ°ÑÐ¸ÐºÐ° Ð¾ÑÑÑÐµÑÑÐ²Ð»ÑÐµÑÑÑ Ð¿ÑÑÐµÐ¼ Ð¾ÑÑÐ»ÐµÐ¶Ð¸Ð²Ð°Ð½Ð¸Ñ Ð¸Ð·Ð¼ÐµÐ½ÐµÐ½Ð¸Ð¹ Ð² ÐºÐ¾Ð½ÑÐµÐ½ÑÐµ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ Ð¿ÐµÑÑÐµÐ¿ÑÐ¸Ð²Ð½Ð¾Ð³Ð¾ ÑÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ð¹. ÐÐ»Ñ ÑÑÐºÐ¾ÑÐµÐ½Ð¸Ñ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð¸ ÑÐºÐ¾Ð½Ð¾Ð¼Ð¸Ð¸ ÑÑÐ°ÑÐ¸ÐºÐ° ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼ Ð²ÑÐ±Ð¾ÑÐ¾ÑÐ½Ð¾Ð¹ Ð·Ð°Ð³ÑÑÐ·ÐºÐ¸ ÑÐµÑÑÑÑÐ¾Ð². Ð¢ÐµÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¾ Ð½Ð° ÑÐ°Ð·Ð¼ÐµÑÐµÐ½Ð½Ð¾Ð¼ Ð½Ð°Ð±Ð¾ÑÐµ ÑÑÑÐ»Ð¾Ðº Ð¾Ñ ÐºÐ¾Ð¼Ð¿Ð°Ð½Ð¸Ð¸ Â«VKÂ». ÐÐ¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ñ ÐºÑÐ¸ÑÐµÑÐ¸Ð¸ ÐºÐ°ÑÐµÑÑÐ²Ð° Ð²ÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑÐ° Ð¸ Ð¿Ð°ÑÐ°Ð¼ÐµÑÑÑ Ð½Ð°ÑÑÑÐ¾Ð¹ÐºÐ¸ ÑÐ¸ÑÑÐµÐ¼Ñ. ÐÐµÐ±-ÑÐºÑÐ°Ð¿ÐµÑ Ð¿Ð¾Ð·Ð²Ð¾Ð»ÑÐµÑ Ð²ÐµÑÐ½Ð¾ ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÑÐ¸ÑÐ¾Ð²Ð°ÑÑ 95% Ð¼Ð¾ÑÐµÐ½Ð½Ð¸ÑÐµÑÐºÐ¸Ñ ÑÐ°Ð¹ÑÐ¾Ð² Ð¸ Ð°Ð²ÑÐ¾Ð¼Ð°ÑÐ¸ÑÐµÑÐºÐ¸ Ð¾Ð´Ð¾Ð±ÑÐ¸ÑÑ 75% Ð»ÐµÐ³Ð¸ÑÐ¸Ð¼Ð½ÑÑ ÑÐµÑÑÑÑÐ¾Ð². ÐÐ»Ð³Ð¾ÑÐ¸ÑÐ¼ Ð²ÑÐ±Ð¾ÑÐ¾ÑÐ½Ð¾Ð¹ Ð·Ð°Ð³ÑÑÐ·ÐºÐ¸ ÑÐµÑÑÑÑÐ¾Ð² ÑÐ²ÐµÐ»Ð¸ÑÐ¸Ð²Ð°ÐµÑ Ð¿ÑÐ¾Ð¿ÑÑÐºÐ½ÑÑ ÑÐ¿Ð¾ÑÐ¾Ð±Ð½Ð¾ÑÑÑ ÑÐµÑÐ²Ð¸ÑÐ° Ð½Ð° 25%.
description: This thesis focuses on investigating methods of cloaking on fraudulent websites and developing a methodology for detecting them using a web scraper disguised as a legitimate user. The research objectives are: 1. Reviewing tools for building a web scraper in the context of detecting cloaking websites; 2. Developing algorithms to detect cloaking websites; 3. Creating a web scraper to identify cloaking websites; 4. Testing the developed service and evaluating its effectiveness. To disguise the web scraper, techniques such as IP address rotation, HTTP header spoofing, and emulating device characteristics through browser JavaScript environment configuration are used. Detection of cloaking websites is achieved by monitoring content changes using perceptual hashing of images. To accelerate processing and conserve bandwidth, an algorithm for selective resource loading has been developed. Testing was conducted on a labeled set of links provided by company "VK". Quality criteria for the web scraper and system configuration parameters were determined. The resulting configuration of the web scraper allows for accurate classification of 95% of fraudulent websites and automatic approval of 75% of legitimate resources. The selective resource loading algorithm increases the service throughput by 25%.