Here’s an enhanced version of your prompt with clearer structure, comprehensive explanations, and emphasis on key aspects to make it more accessible to an LLM:

---

### Context:

You have several large CSV databases (each over 10GB), where each database has a column labeled `TC`, which is the unique identifier for each person's data. In every CSV, `TC` values are distinct and appear only once. Your goal is to merge the data for each individual by filling gaps from other databases, ensuring the resulting file is as complete and accurate as possible.

### Key Requirements:

1. **Main Purpose**:
   The goal is to combine data from multiple CSV files based on the `TC` column. The main database should be enriched with data from secondary databases, prioritizing the most accurate data. If a more accurate value for a cell is found in a secondary file, it will overwrite the less accurate one. However, missing values in the main file should be filled using the best available data from secondary files to leave as few empty cells as possible.

2. **General Outline**:
   You want a Python script that merges these databases based on the `TC` identifier, allowing control via user inputs and in-code configurations. This script should work efficiently without loading the entire databases into memory, processing the data line by line or in chunks.

---

### Instructions for the LLM to Generate the Script:

1. **File Handling**:
   - The script will merge several CSV files by adding data from secondary databases to the main database.
   - The output should be a separate CSV file without modifying the input files.
   
2. **TC Column**:
   - `TC` is the unique identifier and will be used to merge rows across databases.
   - The script should add new `TC` values from secondary databases to the main file, leaving other columns for those rows empty initially.
   - This operation does not require user input; it’s handled consistently.

3. **Handling Columns**:
   - The column names across the files may vary. The script must identify all unique column names across all databases (including the main one) and show which columns are present in each file.
   - The user will have control over how to handle each column:
     - **Keep the original column** in the main database as it is.
     - **Combine values from multiple files**, prioritizing the main database’s values, but allowing for a prioritization order across files.
     - **Fill missing cells** in the main database using values from secondary files, according to the prioritization order.
   - The script should allow columns from secondary databases to be included, even if they don’t exist in the main file.
   - Because the actions for the TC column is already explained above, you won't ask the user for his choice for the TC column. Sİnce It is always the same action for every case.

4. **Column Exclusion**:
   - The user should have the option to **delete columns entirely** from the final output, even if they exist in the main database.

5. **Column Renaming and Reordering**:
   - After making decisions about which columns to keep and how to combine them, the script should provide an option to rename and reorder the columns in the final output.

6. **Data Format Handling**:
   - Some CSV files may include values enclosed in apostrophes (e.g., `"value"`) or triple quotes (`"""value"""`). The script should handle both cases while ensuring the final output doesn’t contain unnecessary quotes, except when needed (e.g., for values containing commas).

7. **Memory Efficiency**:
   - Since the files are large, the script must process data **line by line** or in **chunks**, avoiding full file loads into memory. Use efficient libraries and strategies to optimize speed and memory usage.

8. **Logging and Feedback**:
   - The script should print detailed progress at each step (e.g., when combining columns, handling missing data).
   - A log file should be created to capture all relevant processing details.
   - Use `tqdm` to show progress bars for any process (e.g., file reading, row processing) and include file sizes or line counts to estimate the time remaining.

9. **Configuration Saving**:
   - After the user makes their column-combining choices, the script should ask if the configuration should be saved.
   - The script should store file names and settings in the configuration. If the same files are loaded later, the script should offer the user the option to reuse the saved configuration.

10. **Output Preview**:
   - After processing the column data and before saving the output file, the script should print a structured preview of the final CSV’s columns, including the prioritization of data sources for each column.
   - This preview should be as visually clear as possible.

11. **Code Explanation**:
   - When generating the Python code, explain each feature and implementation detail, including the rationale for decisions (e.g., why certain libraries were used for efficiency).
   - Provide all code at once. If there’s too much to fit in one response, continue with the next portion in the following message.

12. **CSV size limit**:
   - When working with these types of cases you need to set an appropriate CSV size limit and sometimes setting it to the max directly will will make it raise different errors. Keep in mind if you decide to use csv library even though I would like you not to use the builitin csv library. I think you should use polars library to handle large csv files.

13. **Database names**:
   - I need you to use this as the files part in the script. SO you don't get the file paths through user inputs but rather from inside the script directly.


parent_folder = os.path.abspath(os.path.join(os.getcwd(), '..'))

# Specify the file paths directly here
MAIN_FILE = os.path.join(parent_folder, 'Ultimate_Database', 'final_database_v2.1_sorted.csv')
SIDE_FILES = [
    os.path.join(parent_folder, 'Data Dump', '109m.csv')
]

OUTPUT_FILE = 'Actual_Ultimate_Database.csv'
CONFIG_FILE = 'merge_config.json'
LOG_FILE = 'merge_log.txt'


---

### Sample CSV Formats:

- **Input Format with Apostrophes**:
   ```
   "Id","TC","ADI","SOYADI","ANAADI","BABAADI","DOGUMYERI","DOGUMTARIHI"
   1,18445070762,"AHMET","AYDOĞDU","AYŞE","ALİ","AKÖREN",1936-07-01
   2,18442070826,"DURSUN","AYDOĞDU","ANIŞ","MEHMET","KARAİSALI",1938-01-01
   ```

- **Desired Output Format**:
   ```
   TC,AD,SOYAD,GSM,BABAADI,BABATC,ANNEADI,ANNETC,DOGUMTARIHI,OLUMTARIHI
   10121336568,BİN,KOÇ,,MEHMET,,ŞAHER,,1971-10-01
   19184111174,FATİH,DAĞAOTURAN,5327328676,AHMET,19214110122,HATİCE,19199110674,1977-09-10
   ```

---