Update README.md

README.md

@@ -13,25 +13,96 @@ pinned: false
 ## Deployment
 [HuggingFace](https://huggingface.co/spaces/AtharvaThakur/Insights)
 
-## Modules
+# Insights: Gen-AI Based Data Analysis Tool
 
-- `DataLoader`: Handles the loading of data either by uploading a CSV file or inputting a URL to a CSV file.
-- `DataAnalyzer`: Provides summary statistics and data types of the loaded dataset.
-- `DataFilter`: Allows users to filter rows based on user-defined conditions.
-- `DataTransformer`: Enables users to perform operations on columns.
-- `DataVisualizer`: Visualizes data with various types of plots (Histogram, Box Plot, Pie Chart, Scatter Plot, Heatmap).
+## Overview
+
+**Insights** is a state-of-the-art data analysis tool that leverages the Gemini-Pro large language model (LLM) to automate and enhance the data analysis process. This tool aims to perform end-to-end data analysis tasks, providing substantial cost and time savings while matching or exceeding the performance of junior data analysts.
+
+## Table of Contents
+
+1. [Introduction](#introduction)
+2. [Features](#features)
+3. [System Architecture](#system-architecture)
+4. [Modules Overview](#modules-overview)
+5. [Installation](#installation)
+6. [Usage](#usage)
+7. [Evaluation](#evaluation)
+8. [Contributors](#contributors)
+9. [License](#license)
+
+## Introduction
+
+In today's data-driven world, robust data analysis tools are crucial for informed decision-making and strategic planning. Traditional data analysis methods often face challenges such as time-consuming processes, potential for errors, and the need for specialized expertise. **Insights** addresses these issues by utilizing AI to streamline and enhance the data analysis process.
 
 ## Features
 
-- Upload CSV files or load data from a URL.
-- Display the uploaded dataset.
-- Show summary statistics and data types.
-- Filter rows based on user-defined conditions.
-- Perform operations on columns.
-- Visualize data with various types of plots (Histogram, Box Plot, Pie Chart, Scatter Plot, Heatmap).
-- Transform data.
+- **Automated Data Analysis**: Perform data collection, visualization, and analysis with minimal human intervention.
+- **Advanced Summarization**: Generate detailed summaries and potential questions for datasets.
+- **Exploratory Data Analysis (EDA)**: Tools for statistical summaries, distribution plots, and correlation matrices.
+- **Data Cleaning and Transformation**: Functions for handling missing values, outlier detection, normalization, and feature engineering.
+- **Machine Learning Toolkit**: Automates model selection, training, hyperparameter tuning, and evaluation.
+- **Query Answering Module**: Generate Python code to answer user queries and produce visualizations.
+
+## System Architecture
+
+The **Insights** tool is built on the Gemini platform and consists of three main components:
+
+1. **Summary Module**
+2. **QA Module**
+3. **Code Execution and Analysis Generation**
+
+### Summary Module
+
+Extracts essential details about the dataset and generates a comprehensive summary along with potential questions for further exploration.
+
+### QA Module
+
+Handles user queries related to the dataset, generating Python code to answer the queries and produce visualizations.
+
+### Code Execution and Analysis Generation
+
+Executes the generated Python code offline to ensure data security, producing detailed responses and visualizations.
+
+## Modules Overview
 
-## Detailed Installation Instructions
+### Summary Generation
+
+1. **Information Extraction**: Extracts critical information from the dataset.
+2. **Prompting Gemini**: Constructs a detailed prompt for Gemini to generate summaries and questions.
+3. **Summary and Question Generation**: Generates a summary and potential questions for user review.
+
+### Data Exploration
+
+Includes tools for EDA, data cleaning, and data transformation.
+
+### ML Toolkit
+
+Facilitates the creation and evaluation of machine learning models on the dataset.
+
+### QA Module
+
+Allows users to query the dataset and receive answers along with visualizations. The process involves:
+
+1. Accepting user queries.
+2. Combining queries with dataset information.
+3. Generating and executing Python code offline.
+4. Producing visualizations and textual data.
+
+### Analysis Generation
+
+Processes the output from code execution to create concise and insightful responses.
+
+## Usage
+1. Initialize the Tool:
+`python app.py`
+2. Load Dataset: Upload your dataset when prompted.
+3. Generate Summary: The tool will automatically generate a summary and potential questions.
+4. Exploratory Data Analysis: Use the EDA tools to explore your dataset.
+5. Query the Dataset: Enter your queries to receive answers and visualizations.
+6. Analyze Results: Review the detailed analysis generated by the tool.
+
+## Installation Instructions
 
 1. Install the required packages:
    The project's dependencies are listed in the 'requirements.txt' file. You can install all of them using pip:
@@ -44,28 +115,6 @@ pinned: false
    streamlit run app.py
    ```
 
-## Web app
-1. Main page
-   Data Exploration
-   -> Data Loader
-   -> DataQA (LLM with python interpreter/CSV agent)
-   -> Data Analyzer
-   -> Data Filter
-   -> Data Visualizer
-
-2. Data Transformation
-   -> handling null values
-   -> creating new columns
-   -> removing columns
-   -> Changing datatypes
-   -> give option to analyse the transformed dataset or save it.
-
-3. Natural language dataparty (Pure LLM)
-   -> Insights generation
-   -> Automating the data analysis/transformation
-   -> generating a report
-
-
 # Running using Docker
 1. Build the docker image using 
    ```