Need to create a telegram bot that will parse data from the upwork exchange.

General information Telegram-bot is designed for convenient search of orders on Upwork exchange. It provides flexible filter settings, notifications about new orders, detailed information about clients and an intuitive interface. All interactions with the bot occur via buttons placed under the messages, which makes the use of the bot as simple and convenient as possible. Functionality 1. Getting started with the bot - The user launches the bot, receives a welcome message and presses the “Start” button. - The bot sends a brief description of its function and prompts the user to customize the order search settings, providing buttons to select the next action. 2. Customizing Search Filters The bot provides a step-by-step system for selecting parameters to filter orders. Each selection step is accomplished using the buttons below the messages. 1. Order Category - The user selects an order category (e.g. Design & Creative, Development, Marketing & Advertising) via the buttons displayed in the message. - After selecting a category, the user sees buttons to specify subtopics such as: - For Design: “Graphic Design”, “2D Animation”, “Video Making”. - For development: “Web development”, “Mobile applications”, “API integrations”. 2. Minimum Budget - The Bot prompts you to choose a minimum budget from the options offered (e.g. $50, $100, $500) via buttons. 3. Experience Level - The user selects a skill level via buttons: - Entry Level. - Intermediate. - Expert. 4. Job Type - The bot offers two options to choose from: - Hourly (Hourly). - Fixed Price. 5. Client Location - The user can select a country or region from the list presented in the form of buttons. 6. Client's Minimum Spend - The bot offers several options for client's minimum spend on Upwork (e.g. $100, $1000, $5000) which can be selected via buttons. 7. Number of Proposals (Proposals) - The user selects the maximum number of proposals already submitted for the order, e.g. less than 5, 10 or 20. 8. Publication Date - The bot offers to select a period of time since the order was published: - Last 24 hours. - Last 3 days. - Last week. 9. Required Skills - User selects key skills from a suggested list via buttons (e.g. Python, Adobe Photoshop, SEO). 10. Language - The user selects the language to communicate or do the work (e.g. English, French, Spanish) also via buttons. 11. Payment Verification - The user can select the filter “Only clients with verified payment information” through the button. 3. Working with notifications The bot promptly notifies users about new orders using buttons to interact with: 1. Periodic update - Every 5 minutes the bot updates the list of available orders taking into account the specified filters. - Notifications contain the following data: - Order name. - Short description. - Budget. - Type of payment (hourly or fixed). - Name of the customer. - Link to the customer's profile. - Total amount spent by the customer on Upwork. - Link to the order. 2. Customer information - Notifications provide buttons to go to: - To order. - To the customer's profile. 3. Disable Notifications feature - The user can suspend notifications by clicking the “Disable Notifications” button. - Once activated, notifications can be turned back on via the “Enable Notifications” button. 4. Notification Management - Each notification message contains buttons for: - View order details. - Remove an order from notifications if it is not interesting. 5. Sending only new orders - The bot eliminates duplicate orders that have already been sent to the user previously. Ensuring reliability 1. Protection against duplicate notifications - The bot tracks the history of sent orders for each user, eliminating duplicate notifications. 2. Backup migration - In case of failures, the bot automatically migrates work to a backup server, ensuring continuity. Key Features - Intuitive interface with buttons under each message. - Highly flexible filter customization. - Prompt notifications of new orders only. - Complete information about orders and customers. - Easy to manage notifications via buttons.

Technical description of Telegram-bot with the possibility of adaptation to Discord The logic of bot operation The bot is based on three key principles: 1. Universality of the core: The basic logic of parsing and filtering data is independent of the platform. 2. Adapter flexibility: The platform-dependent interaction layer (Telegram/Discord) is separated from the core. 3. Resource optimization: Use of user groups, caching and asynchronous task processing. 1. Main work stages Stage 1: User registration 1. The user interacts with the bot via Telegram. - Input data: Telegram ID, selected filters. - Actions: - Check if there is a group with these filters. - If the group exists: - The user is added to the group. - If there is no group: - A new group is created. - The corresponding parsing line is activated. - Outgoing data: Group ID, updated structure of the groups collection. Step 2: Customizing filters 1. User selects filter parameters via inline buttons: - Category, minimum budget, experience level, job type, etc. - Input: Filter settings from the user. - Actions: - Saves the selected filters in the database. - Checks if a group with those filters exists. - Outgoing data: - Updated data of the users collection. - If the group does not exist, a new entry in groups is created. Step 3: Activate parsing 1. If a new group is created, the parser activates the line for its filters: - Incoming data: Group Filters. - Actions: - Start parsing orders from Upwork. - Caching parsing data in Redis. - Saving relevant orders to the database. - Outbound data: Records in the orders collection. Stage 4: User Notifications 1. New orders are sent to group members: - Incoming data: Orders from the database. group users. - Actions: - Grouping orders. - Check if orders have been sent before. - Sending notifications to all group members. - Outgoing data: Messages via Telegram API. Stage 5: Periodic maintenance 1. Outdated orders and inactive groups are deleted: - Inbound data: Order creation time and user activity. - Actions: - Deleting entries from orders older than 24 hours. - Disable parsing lines for groups without members. - Outgoing data: Updates in orders and groups collections. 2. Bot Architecture 1. Main components - Core logic: - Responsible for data parsing, order filtering, user management and notifications. - Independent of Telegram and Discord, allows easy adaptation of the logic to other platforms. - Platform-dependent adapters: - Telegram-adapter: Handling inline buttons, CallbackQuery, messages. - Discord-adapter: Slash-commands, interactive buttons. 2. Project structure bot/ ├── core/ # Core logic │ ├── filters.py # Logic of working with filters │ ├── upwork_parser.py # Upwork parser │ ├── notifications.py # User notifications │ ├── group_manager. py # User group management │ └─── utils.py # Auxiliary functions ├── telegram_adapter/ # Telegram adapter │ ├── main. py # Entry point for Telegram │ ├── handlers.py # Command and button handlers │ ├── telegram_api.py # Telegram API │ └─── buttons. py # Generate inline buttons ├── discord_adapter/ # Discord adapter (similar to Telegram) │ ├── main.py # Entry point for Discord │ ├── handlers. py # Handlers for commands and buttons │ ├── discord_api.py # Discord API │ └─── buttons.py # Generating interactive elements ├── db/ # Working with database │ ├── database. py # Connect to MongoDB │ ├── users.py # Manage users │ ├── orders.py # Manage orders │ ├── groups.py # Manage groups ├── config.py # Configuration (API keys, database URL) └─── requirements.txt # Dependencies 3. Incoming and outgoing data 1. Incoming data - From the user: - IDs, filters, commands. - From the parser: - New orders. - From the database: - Lists of groups, users, notifications sent. 2. Outbound data - To user: - Order messages. - To database: - Filter updates, group creation, order saving. - To platform (Telegram/Discord): - Notifications and responses to commands. 4. Logic of adaptation for Discord 1. Processing of incoming data: - Telegram: Inline buttons, CallbackQuery. - Discord: Slash-commands, interactive buttons. 2. Sending messages: - A common notification format is used for both platforms. - Adapters convert data to the appropriate format. 3. Groups and filters: - The logic of user aggregation and parsing remains unchanged. - Only adapters call platform specific kernel functions. Operating Logic Stage 1: User Registration 1. The user interacts with the bot via Telegram by clicking the “Start” button. 2. The bot asks for filter settings through a sequential chain of questions: - Category. - Budget. - Experience level. - Job type, etc. 3. After selecting the filters: - The existence of a group with similar filters is checked. - If the group exists, the user is added to it. - If the group does not exist: - A new group is created. - The parsing line for the filters of this group is activated. Step 2: Working with filters 1. User filters are saved in the database (users). 2. If several users have selected the same filters: - They are combined into one group. - Notifications are sent to the group instead of individually to each user. Step 3: Parsing the data 1. The parser activates lines only for the filters in use: - For example, if a group has selected the filter category=Design, budget_min=100, parsing is performed only on these criteria. 2. Data is stored in cache (Redis) to minimize latency. 3. Relevant data from the cache is saved to the database (orders) where it will be available for sending notifications. Step 4: Notifications 1. After saving a new order to orders, the bot checks which groups it is relevant for. 2. Notifications are sent to all members of the group: - If a user has disabled notifications, the order is not sent to them. 3. The fact that a notification is sent is logged to avoid duplication. Step 5: Periodic Maintenance 1. Outdated orders (older than 24 hours) are deleted. 2. Groups without members are deactivated: - Their parsing lines are stopped. Database structure The database is designed based on MongoDB, but can be adapted for PostgreSQL if needed. Collection: users Stores data about users and their filter settings. Document structure: { “_id”: “123456789”, // Unique user ID (Telegram/Discord ID) “platform”: “telegram”, // Platform (telegram or discord) “username”: “JohnDoe”, // Username “filters”: { // User filter settings “category”: “Design”, “budget_min”: 100, “experience_level”: “Intermediate”, ‘work_type’: ‘Fixed Price’, ‘client_location’: [“USA”, “UK”], // Client geography “skills”: [“Photoshop”] // Skills }, “group_id”: “group_1”, // ID of the group the user belongs to “notifications_enabled”: true, // Notification status “last_activity”: “2024-11-17T10:00:00:00Z” // Last activity } Indexing: - Fields: _id, group_id, last_activity. Collection: groups Stores information about user groups. Document structure: { “_id”: “123456789”, // Unique user ID (Telegram/Discord ID) “platform”: “telegram”, // Platform (telegram or discord) “username”: “JohnDoe”, // User name “filters”: { // User filter settings “category”: “Design”, “budget_min”: 100, “experience_level”: “Intermediate”, ‘work_type’: ‘Fixed Price’, ‘client_location’: [“USA”, “UK”], // Client geography “skills”: [“Photoshop”] // Skills }, “group_id”: “group_1”, // ID of the group the user belongs to “notifications_enabled”: true, // Notification status “last_activity”: “2024-11-17T10:00:00:00Z” // Last activity } Indexing: - Fields: _id, group_id, last_activity. Collection: groups Stores information about user groups. Document structure: { “_id”: “group_1”, // Unique group identifier “filters”: { // Filters active for the group “category”: “Design”, “budget_min”: 100, “experience_level”: “Intermediate”, ‘work_type’: ‘Fixed Price’, ‘client_location’: [ “USA”, “UK” ], “skills”: [“Photoshop” ] }, “user_ids”: [“123456789”, “987654321”], // List of users in the group “parsing_active”: true, // Activation status of parsing line “created_at”: “2024-11-16T09:00:00:00Z”, // Group creation date ‘last_parsed’: ‘2024-11-17T10:00:00:00Z’ // Last parsing time } Indexing: - Fields: _id, filters.category, filters.budget_min. Collection: orders Stores information about orders matching group filters. Document structure: { “_id”: “order_987654321”, // Unique order identifier “group_id”: “group_1”, // ID of the group for which the order is relevant “title”: “Create a logo design”, // Order title ‘description’: “We need a creative logo...”, // Brief description ‘budget’: 500, // Budget “type”: “Fixed Price”, // Job type “client”: { // Client info “name”: “JaneDoe”, “profile_url”: “https://upwork.com/profile/12345”, ‘spent’: 2000 }, “skills”: [ “Photoshop”, “Design” ], // Required Skills “created_at”: “2024-11-17T09:00:00:00Z”, // Publication date ‘expires_at’: “2024-11-18T09:00:00:00Z”, // Retention period ‘order_url’: “https://upwork.com/job/12345” // Link to order } Indexing: - Fields: group_id, created_at, expires_at. Collection: notifications_log Stores information about which orders have been sent to users. Document structure: { “user_id”: “123456789”, // user ID “order_id”: “order_987654321”, // order ID “sent_at”: “2024-11-17T10:05:00Z” // Date and time the notification was sent } Indexing: - Fields: user_id, order_id. Algorithm of operation 1. Registration 1. User selects filters. 2. A group is created or added to an existing group. 3. Parsing line for new groups is activated. 2. Parsing 1. The parser retrieves active lines from the groups collection. 2. Extracts data, saves it to Redis for speedup. 3. Relevant orders are stored in orders. 3. Notifications 1. New orders are checked for each group. 2. Notifications are sent to the members of the group. 3. The notifications sent are logged. 4. Deleting old data 1. Orders are deleted from orders whose retention period has expired. 2. Groups without users are marked as inactive.