///
# CRITICALLY IMPORTANT RULES
1. Completeness: Generate full code, no placeholders. If unable, explain in comments.
2. Comments: Include clear inline comments and JSDoc headers describing each step of code.
3. Error Checking: Implement error checking and type validation.
4. Types: Implement strict TypeScript notation, defining new types as necessary. Additionally:
   - Do not use the 'any' type.
   - Do not use the non-null assertion operator (!).
   - Do not cast to unknown (e.g. as unknown as T).
5. Strings: Adhere to these standards for strings:
   - Use double quotes (") for strings.
   - Use string templates or .join() instead of operational concatenation.
It is critically important that you adhere to the above five rules. ///







Your aim is to develop a robust and feature-rich text editor tailored for poets and lyricists working in both English and Russian. This editor should highlight vowels and similar word combinations with consistent and unique colors, display vowel counts at the end of each line, and offer a user-friendly interface with customizable settings.

Below is a comprehensive, step-by-step plan to rewrite and enhance the text editor, ensuring it meets all your specified requirements and provides a smooth user experience.

I. Project Overview
Objective:
Create a Python-based text editor using Tkinter that:

Highlights vowels (English and Russian) with consistent colors.
Highlights similar word combinations (e.g., common substrings) with unique colors.
Displays vowel counts at the end of each line.
Offers user controls for customization, such as toggling highlights and changing background colors.
Ensures a dark-themed user interface for better readability.
Key Features:
Vowel Highlighting: Both English and Russian vowels are highlighted consistently throughout the text.
Word Combination Highlighting: Similar substrings within words are identified and highlighted with unique colors.
Vowel Counts: Each line displays the count of vowels present.
User Controls: Buttons to highlight text, clear highlights, toggle highlighting features, and change background colors.
Robust Tag Management: Efficient handling of Tkinter tags to avoid errors and ensure performance.
Error Handling: Gracefully manages potential errors, especially related to tag configurations.
Responsive UI: Ensures the editor remains responsive even with large texts.
II. Detailed Step-by-Step Plan
1. Set Up the Project Environment
Python Installation: Ensure Python 3.x is installed on your machine.
Tkinter Verification: Tkinter typically comes bundled with Python. Verify by running:

python -c "import tkinter"
If no errors appear, Tkinter is installed.
Dependencies: The project primarily relies on Tkinter and Python's standard libraries (re, collections, etc.), so no additional installations are required.
2. Define Constants and Configuration
Vowels Definition:
English Vowels: AEIOUYaeiouy
Russian Vowels: АЕЁИОУЫЭЮЯаеёиоуыэюя
Combined Vowels: Concatenate both for unified processing.
Color Palettes:
Vowel Colors: Utilize a light color palette to distinguish vowels without overshadowing the main text.
Word Combination Colors: Use a more vibrant palette to make similar parts stand out distinctly.
Ensure Enough Colors: Both palettes should have sufficient colors to handle multiple unique patterns without repetition.
3. Design the User Interface (UI) Layout
Main Components:
Text Area: Central area for text input and editing.
Control Panel: Contains buttons and checkboxes for user interactions.
Counters Display: Shows vowel counts at the end of each line.
UI Structure:
Tkinter Frames: Use frames to organize the layout logically.
ScrolledText Widget: Utilize ScrolledText for the main text area to provide scrolling functionality.
Buttons and Checkboxes: Place control elements in a dedicated frame below or beside the text area.
Background Theme: Apply a dark background to the text area and control panels for reduced eye strain.
User Controls:
Highlight Button: Triggers the highlighting of vowels and word combinations.
Clear Button: Removes all highlights and counts.
Checkboxes: Allow users to toggle the highlighting of vowels and word combinations independently.
Background Color Selector: Dropdown menu to let users choose different background colors.
4. Implement the Text Area with Scrollbars
Widget Selection: Use ScrolledText.ScrolledText for the text area to inherently provide vertical scrolling.
Configuration:
Fonts: Choose a monospaced font like "Consolas" for better alignment and readability.
Background and Foreground Colors: Set the text area's background to a dark color (e.g., #1e1e1e) and text color to a light shade (e.g., #d4d4d4).
Insert Cursor: Configure the cursor's color to be prominent (e.g., white).
5. Manage Tag Configurations for Highlighting
Understanding Tags:
Tkinter Tags: Used to apply styles (like foreground color) to specific portions of text.
Dynamic Tag Creation: Tags must be created dynamically based on the patterns detected (vowels and word combinations).
Tag Naming Strategy:
Unique Identifiers: Use a combination of a prefix and a hashed representation of the pattern to create unique and safe tag names.
Avoid Special Characters: This prevents issues with tags containing non-ASCII or special characters.
Example Tag Name Generation:

import hashlib

def generate_tag_name(prefix, content):
    content_hash = hashlib.md5(content.encode('utf-8')).hexdigest()
    return f"{prefix}_{content_hash}"
Tag Configuration:
Vowels:
Assign a unique color from the vowel color palette.
Tags are named using the vowel character and its hash.
Word Combinations:
Assign a unique color from the word combination color palette.
Tags are named using the substring and its hash.
Error Handling:
Tag Existence Check: Before retrieving or configuring a tag, ensure it exists to prevent TclError.
Graceful Failures: Wrap tag operations in try-except blocks or perform existence checks.
6. Implement Vowel Highlighting
Regex Pattern:
Use a regex pattern that matches any vowel (both English and Russian).
Example:

VOWEL_PATTERN = '[' + re.escape(ALL_VOWELS) + ']+'
Highlighting Process:
Retrieve Text: Get the entire text from the text area.
Find Matches: Use re.finditer with the vowel pattern to locate all vowels and their sequences.
Assign Colors:
For each unique vowel sequence, assign a color from the vowel color palette.
Use the generate_tag_name function to create a unique tag.
Apply Tags: Use tag_add to apply the tag to the matched text range.
7. Implement Word Combination Highlighting
Objective: Identify and highlight similar substrings within words to recognize patterns like rhymes or common prefixes/suffixes.
Approach:
Word Extraction: Use regex to extract all words from the text.

words = re.findall(r'\b\w+\b', text, re.UNICODE)
Identify Common Substrings:
Compare words pairwise to find common substrings of a minimum length (e.g., 2 characters).
Store found substrings in a collection ensuring uniqueness and relevance.
Filter Overlapping Substrings:
Prioritize longer substrings to maximize pattern recognition.
Avoid overlapping highlights to maintain clarity.
Assign Colors:
Assign a unique color from the word combination color palette to each unique substring.
Use the generate_tag_name function to create unique and safe tag names.
Apply Tags: Use tag_add to apply the tag to the matched substring ranges.
Optimization:
Avoid Redundancy: Ensure that the same substring isn't processed multiple times.
Performance: Implement efficient algorithms to handle large texts without significant delays.
8. Display Vowel Counts per Line
Objective: Count the number of vowels in each line and display the count at the end.
Implementation Steps:
Retrieve Text: Get the entire text and split it into lines.
Count Vowels: For each line, count the number of vowels using a generator expression.

count = sum(1 for char in line if char in ALL_VOWELS)
Display Counts:
Insert the count at the end of each line in a specific format (e.g., [5]).
Apply a distinct tag to the count text to style it differently (e.g., darker gray color).
Manage Counts:
Ensure counts are updated dynamically as the text changes.
Prevent duplicate counts by removing previous count tags before updating.
Edge Cases:
Empty Lines: Handle lines with no content gracefully.
Line Endings: Ensure counts are correctly inserted without disrupting the text structure.
9. Implement Control Panel with User Interaction Elements
Features:
Highlight Button: Initiates the highlighting process based on selected options.
Clear Button: Removes all highlights and counts, resetting the text area.
Checkboxes:
Highlight Vowels: Toggle vowel highlighting on or off.
Highlight Similar Parts: Toggle word combination highlighting on or off.
Background Color Selector:
Dropdown menu allowing users to choose different background colors for the text area.
Layout:
Place the control panel below or beside the text area for easy access.
Use padding and spacing to ensure buttons and checkboxes are neatly arranged.
Styling:
Buttons: Use distinct colors to indicate actions (e.g., green for highlight, red for clear).
Checkboxes: Clearly label to indicate their functionality.
10. Bind Events to Enable Dynamic Highlighting and Counting
Text Modification Event:
Bind the <<Modified>> event of the ScrolledText widget to a callback function that triggers highlighting and counting.
Example:

self.text_area.bind('<<Modified>>', self.on_text_modified)
Callback Function (on_text_modified):
Reset Modified Flag: Prevent recursive calls by resetting the modified flag.

self.text_area.edit_modified(False)
Trigger Highlighting:
Check the state of the checkboxes to determine which highlighting features to apply.
Call the respective highlighting functions (highlight_vowels, highlight_similar_parts).
Update Vowel Counts:
After highlighting, update the vowel counts per line.
Performance Considerations:
Debouncing: Implement debouncing to prevent excessive processing during rapid text changes.
Efficient Algorithms: Use optimized regex patterns and substring matching algorithms to maintain responsiveness.
11. Implement Robust Tag Management
Tag Storage:
Maintain dictionaries to map patterns (vowels and substrings) to their assigned colors.
Example:

self.vowel_colors = {}
self.similar_part_tags = set()
Tag Clearing:
Before applying new highlights, remove existing tags to prevent overlapping or outdated highlights.
Example:

self.text_area.tag_remove(tag, "1.0", tk.END)
Avoid Tag Duplication:
Use unique and consistent tag names to ensure the same patterns receive the same tags and colors.
12. Ensure Error Handling and Stability
Potential Errors:
Undefined Tags: Accessing or modifying undefined tags can raise TclError.
Invalid Regex Patterns: Malformed patterns can cause regex operations to fail.
Solutions:
Tag Existence Checks: Before configuring or accessing a tag, verify its existence.
Try-Except Blocks: Wrap tag operations in try-except blocks to catch and handle exceptions gracefully.
Input Validation: Ensure that user inputs (if any) for patterns or colors are valid.
Example Error Handling:

if tag_name not in self.text_area.tag_names():
    self.text_area.tag_configure(tag_name, foreground=assigned_color)
13. Optimize Performance for Large Texts
Efficient Processing:
Limit the scope of regex searches to recently modified parts of the text.
Use compiled regex patterns to speed up repeated searches.
Resource Management:
Avoid excessive creation of tags by reusing existing ones where possible.
Implement lazy loading or incremental processing for very large documents.
User Feedback:
Provide visual indicators (like a loading spinner) if processing is time-consuming.
Allow users to pause or cancel intensive operations.
14. Testing and Validation
Unit Testing:
Test individual functions like highlight_vowels and highlight_similar_parts with various input scenarios.
Integration Testing:
Ensure that components like event bindings, tag management, and UI elements work seamlessly together.
User Testing:
Test the editor with actual poetry and lyrics to validate the highlighting and counting functionalities.
Gather feedback on usability and adjust the UI accordingly.
Edge Cases:
Test with empty lines, lines with no vowels, lines with repetitive vowels, and lines with complex word combinations.
Test with mixed-language texts (English and Russian) to ensure accurate highlighting.
15. Documentation and User Guidance
Code Documentation:
Comment key sections of the code to explain functionality and logic.
Provide docstrings for classes and functions.
User Instructions:
Create a help menu or tooltip explanations for buttons and checkboxes.
Include guidance on how to use the editor's features effectively.
16. Future Enhancements (Optional)
File Operations:
Implement functionality to open and save text files.
Custom Pattern Highlighting:
Allow users to define and highlight their own regex patterns.
Theme Customization:
Provide multiple color themes beyond the default dark background.
Exporting Highlights:
Enable exporting the highlighted text with preserved formatting to formats like HTML or PDF.
Real-Time Collaboration:
Explore integrating real-time collaboration features for multiple users to edit simultaneously.
III. Implementation Guidelines
To facilitate the development process, here's a breakdown of how to implement some of the key components based on the above plan.

A. Setting Up the Main Application Class
Create the Main Class:

class EnhancedTextEditor:
    def __init__(self, root):
        self.root = root
        self.root.title("Enhanced Text Editor with Pattern Recognition")
        self.setup_ui()
        self.initialize_tags()
Set Up the UI:

def setup_ui(self):
    # Create main frame
    self.main_frame = tk.Frame(self.root, bg="#2e2e2e")
    self.main_frame.pack(fill=tk.BOTH, expand=True)

    # Initialize text area
    self.text_area = scrolledtext.ScrolledText(
        self.main_frame,
        wrap=tk.WORD,
        font=("Consolas", 12),
        bg="#1e1e1e",
        fg="#d4d4d4",
        insertbackground="#ffffff",
        undo=True,
        padx=5,
        pady=5
    )
    self.text_area.pack(fill=tk.BOTH, expand=True)

    # Create control panel
    self.control_panel = tk.Frame(self.root, bg="#2e2e2e")
    self.control_panel.pack(fill=tk.X, padx=10, pady=5)

    # Add buttons and checkboxes as per the plan
    self.add_controls()

    # Bind text modification event
    self.text_area.bind('<<Modified>>', self.on_text_modified)
Adding Controls:

def add_controls(self):
    # Highlight Button
    self.highlight_button = tk.Button(
        self.control_panel,
        text="Highlight Text",
        command=self.highlight_text,
        bg="#4CAF50",
        fg="white",
        padx=10
    )
    self.highlight_button.pack(side=tk.LEFT, padx=5)

    # Clear Button
    self.clear_button = tk.Button(
        self.control_panel,
        text="Clear All",
        command=self.clear_everything,
        bg="lightcoral",
        fg="white",
        padx=10
    )
    self.clear_button.pack(side=tk.LEFT, padx=5)

    # Toggle Checkboxes
    self.highlight_vowels_var = tk.BooleanVar(value=True)
    self.highlight_words_var = tk.BooleanVar(value=True)

    self.vowel_checkbox = tk.Checkbutton(
        self.control_panel,
        text="Highlight Vowels",
        variable=self.highlight_vowels_var,
        bg="#2e2e2e",
        fg="white"
    )
    self.vowel_checkbox.pack(side=tk.LEFT, padx=5)

    self.word_checkbox = tk.Checkbutton(
        self.control_panel,
        text="Highlight Similar Parts",
        variable=self.highlight_words_var,
        bg="#2e2e2e",
        fg="white"
    )
    self.word_checkbox.pack(side=tk.LEFT, padx=5)

    # Background Color Selector
    self.bg_color_var = tk.StringVar(value="#1e1e1e")
    self.bg_color_label = tk.Label(self.control_panel, text="Background:", bg="#2e2e2e", fg="white")
    self.bg_color_label.pack(side=tk.LEFT, padx=5)

    self.bg_color_menu = tk.OptionMenu(
        self.control_panel,
        self.bg_color_var,
        "#1e1e1e", "#262626", "#333333", "#404040",
        command=self.change_background
    )
    self.bg_color_menu.pack(side=tk.LEFT, padx=5)
B. Managing Tag Configurations
Initialize Tag Dictionaries:

def initialize_tags(self):
    # Vowel colors mapping
    self.vowel_colors = {}
    for i, vowel in enumerate(set(ALL_VOWELS)):
        color = VOWEL_COLORS[i % len(VOWEL_COLORS)]
        self.vowel_colors[vowel] = color
        tag_name = self.generate_tag_name("vowel", vowel)
        self.text_area.tag_configure(tag_name, foreground=color)

    # Similar parts set
    self.similar_part_tags = set()

    # Vowel count tags set
    self.vowel_count_tags = set()
Generate Unique Tag Names:

import hashlib

def generate_tag_name(self, prefix, content):
    content_hash = hashlib.md5(content.encode('utf-8')).hexdigest()
    return f"{prefix}_{content_hash}"
C. Implementing Highlighting Functions
Highlight Vowels:

def highlight_vowels(self):
    text = self.text_area.get("1.0", tk.END)
    for vowel, color in self.vowel_colors.items():
        tag_name = self.generate_tag_name("vowel", vowel)
        pattern = re.compile(re.escape(vowel))
        for match in pattern.finditer(text):
            start = f"1.0+{match.start()}c"
            end = f"1.0+{match.end()}c"
            self.text_area.tag_add(tag_name, start, end)
Highlight Similar Parts:

def highlight_similar_parts(self):
    text = self.text_area.get("1.0", tk.END)
    words = re.findall(r'\b\w+\b', text, re.UNICODE)
    
    # Find common substrings of minimum length 2
    substrings = defaultdict(int)
    for word in words:
        word = word.lower()
        for i in range(len(word)):
            for j in range(i+2, len(word)+1):
                substr = word[i:j]
                substrings[substr] += 1

    # Filter substrings occurring more than once
    common_substrings = {k: v for k, v in substrings.items() if v > 1}

    # Sort substrings by length descending to prioritize longer matches
    sorted_substrings = sorted(common_substrings.keys(), key=lambda x: len(x), reverse=True)

    # Assign unique colors to each substring
    substring_color_map = {}
    color_index = 0
    for substr in sorted_substrings:
        if substr not in substring_color_map:
            color = WORD_COMBINATION_COLORS[color_index % len(WORD_COMBINATION_COLORS)]
            substring_color_map[substr] = color
            color_index += 1

    # Highlight substrings
    for substr, color in substring_color_map.items():
        tag_name = self.generate_tag_name("similar", substr)
        self.similar_part_tags.add(tag_name)
        self.text_area.tag_configure(tag_name, background=color)
        for match in re.finditer(re.escape(substr), text, re.IGNORECASE):
            start = f"1.0+{match.start()}c"
            end = f"1.0+{match.end()}c"
            self.text_area.tag_add(tag_name, start, end)
D. Implementing Vowel Count Display
Count Vowels in Each Line:

def count_vowels_in_line(self, line):
    return sum(1 for char in line if char in ALL_VOWELS)
Update Vowel Counts:

def update_vowel_counts(self):
    # Remove existing count displays
    for tag in self.vowel_count_tags:
        self.text_area.tag_remove(tag, "1.0", tk.END)
    self.vowel_count_tags.clear()

    text = self.text_area.get("1.0", tk.END)
    lines = text.split('\n')

    for i, line in enumerate(lines[:-1], 1):  # Skip last empty line
        count = self.count_vowels_in_line(line)
        if count > 0:
            tag_name = f"vowel_count_{i}"
            self.vowel_count_tags.add(tag_name)

            # Configure tag with distinct color
            self.text_area.tag_configure(tag_name, foreground="#808080")  # Dark Gray

            # Insert count at end of line
            line_end = f"{i}.end"
            count_text = f" [{count}]"
            self.text_area.insert(line_end, count_text)

            # Apply tag to count
            count_start = f"{i}.end-{len(count_text)}c"
            count_end = f"{i}.end"
            self.text_area.tag_add(tag_name, count_start, count_end)
E. Implementing Highlighting and Clear Functions
Highlight Text Function:

def highlight_text(self):
    self.clear_all_highlights()
    if self.highlight_vowels_var.get():
        self.highlight_vowels()
    if self.highlight_words_var.get():
        self.highlight_similar_parts()
    self.update_vowel_counts()
Clear All Highlights Function:

def clear_all_highlights(self):
    # Clear vowel highlights
    for vowel in self.vowel_colors.keys():
        tag_name = self.generate_tag_name("vowel", vowel)
        self.text_area.tag_remove(tag_name, "1.0", tk.END)

    # Clear similar parts highlights
    for tag in self.similar_part_tags:
        self.text_area.tag_remove(tag, "1.0", tk.END)
    self.similar_part_tags.clear()

    # Clear vowel counts
    for tag in self.vowel_count_tags:
        self.text_area.tag_remove(tag, "1.0", tk.END)
    self.vowel_count_tags.clear()
F. Implementing Background Color Change
Change Background Function:

def change_background(self, color):
    self.text_area.configure(background=color)
G. Running the Application
Initialize and Run the Editor:

def run(self):
    self.root.mainloop()
H. Handling Text Modification Events
Text Modified Callback:

def on_text_modified(self, event=None):
    if self.text_area.edit_modified():
        self.text_area.edit_modified(False)
        self.highlight_text()