Repository: langflow-ai/langflow
Pull request: feat: Enhance Data Components and Add SelectData Functionality
- **Create Data Component**: 
  - Added **Text Key Validation** to ensure valid key names. If an invalid key is provided, an error will be raised.
  
- **Update/Append Data Component**:
  - Enhanced functionality to allow both updating and appending data. This update supports adding multiple data entries to existing data.
  - Added the ability to update the text key during the update/append process.

- **Text Key Validator**: 
  - Introduced a toggleable `text_key_validator` to allow users to disable key validation when data is dynamically sourced, and key validation is not necessary.

**Fixes**:
- Corrected issues in both the Create Data and Update/Append Data components to improve functionality and reliability.

closes #3644

<img width="911" alt="Screenshot 2024-09-06 at 1 38 39 PM" src="https://github.com/user-attachments/assets/7140a3b2-368c-4578-9d3a-759c8b0d4d6d">


Problem statement:
Create Data and Update Data do not work
### Bug Description

The input nodes of Create Data do not accept any input from any text output. Looking into the code, the inputs of Create Data accept "Text" and "Data". But text outputs from other nodes have the type Message. The same goes for the text input on Update Data.

Connecting a Create Data node with an Update Data node produces the error `Component  field 'data'  might not be a valid input.`, if you press the play button on either node.

### Reproduction

See description.

### Expected behavior

Since the API Request node requires data as input, I would like those nodes to work.

### Who can help?

_No response_

### Operating System

Windows 11

### Langflow Version

v1.0.17

### Python Version

3.12

### Screenshot

_No response_

### Flow File

_No response_

Hints from issue discussion:
<!-- Greeting -->
Hey @Banbury! I'm a bot here to help you with bugs, questions, and becoming a contributor. Let's tackle this issue together while we wait for a human maintainer.

<!-- Answer -->
The `Create Data` and `Update Data` nodes in Langflow v1.0.17 do not accept input from text outputs because they are designed to handle data of type `Data`, not `Message` or `Text`. The error `Component field 'data' might not be a valid input` occurs when connecting a `Create Data` node with an `Update Data` node because the input types do not match the expected data types.

- The `Create Data` node expects input of type `Data`.
- The `Update Data` node also expects input of type `Data`.
- Text outputs, such as those from `ParseData` or `OpenAIModel`, are of type `Message` or `Text`.

Here is an example of the relevant part of the JSON configuration:

```json
{
  "sourceHandle": {
    "dataType": "File",
    "id": "File-31fJd",
    "name": "data",
    "output_types": [
      "Data"
    ]
  },
  "targetHandle": {
    "fieldName": "data",
    "id": "ParseData-rLcoq",
    "inputTypes": [
      "Data"
    ],
    "type": "other"
  }
}
```

This configuration shows that the `data` field expects `Data` type, not `Message` or `Text` <sup>[[1]](https://github.com/langflow-ai/langflow/blob/dev/src/backend/base/langflow/initial_setup/starter_projects/Document QA.json)</sup>.

To resolve this, you may need to use type conversion utilities to convert `Message` types to `Data` types. There are existing functions in the codebase that can help with this conversion <sup>[[2]](https://github.com/langflow-ai/langflow/blob/main/src/backend/base/langflow/schema/message.py)</sup><sup>[[3]](https://github.com/langflow-ai/langflow/blob/dev/src/backend/base/langflow/schema/message.py)</sup><sup>[[4]](https://github.com/langflow-ai/langflow/blob/main/src/backend/base/langflow/services/tracing/utils.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*

@Banbury 

Correct me if I'm wrong, is that what you're trying to do?

Please share the flow that you're trying to create.

<img width="1358" alt="image" src="https://github.com/user-attachments/assets/8b12b1a4-8910-4648-8429-31d44bab250f">

> @Banbury
> 
> Correct me if I'm wrong, is that what you're trying to do?
> 
> Please share the flow that you're trying to create.
> 
> <img alt="image" width="1358" src="https://private-user-images.githubusercontent.com/80289056/363718798-8b12b1a4-8910-4648-8429-31d44bab250f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUzNTg0NzEsIm5iZiI6MTcyNTM1ODE3MSwicGF0aCI6Ii84MDI4OTA1Ni8zNjM3MTg3OTgtOGIxMmIxYTQtODkxMC00NjQ4LTg0MjktMzFkNDRiYWIyNTBmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTAzVDEwMDkzMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTAwNWIzZWMwZWFjNmUzY2ViNWM3ZTU4MWEzMzkxZjFiN2JmMDcyMjVjM2Q4N2I4MzIyNDQzOTU4NGRmMDQ1ZmMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.FwJw_Ebbb8EnqFGxSTKrrwUPiQHNxnzMHz5_K-GtCRo">

how to update data, the 'update data' component's 'New Data' input type is not text, it's a dict, but ,how can we construct a dict?
The Update Data component seems to be outdated. I will direct the team to perform a new update on it soon.

Commit message: Merge branch 'main' into fix-create-data-and-update-data

Patch to review:
diff --git a/src/backend/base/langflow/components/prototypes/CreateData.py b/src/backend/base/langflow/components/prototypes/CreateData.py
index 5ece555edd75..55999f05a9a2 100644
--- a/src/backend/base/langflow/components/prototypes/CreateData.py
+++ b/src/backend/base/langflow/components/prototypes/CreateData.py
@@ -1,7 +1,7 @@
 from typing import Any
 
 from langflow.custom import Component
-from langflow.inputs.inputs import IntInput, MessageTextInput, DictInput
+from langflow.inputs.inputs import IntInput, MessageTextInput, DictInput, BoolInput
 from langflow.io import Output
 
 from langflow.field_typing.range_spec import RangeSpec
@@ -23,7 +23,18 @@ class CreateDataComponent(Component):
             value=0,
             range_spec=RangeSpec(min=1, max=15, step=1, step_type="int"),
         ),
-        MessageTextInput(name="text_key", display_name="Text Key", info="Key to be used as text.", advanced=True),
+        MessageTextInput(
+            name="text_key",
+            display_name="Text Key",
+            info="Key to be used as text.This is useful cause more Parsers/ splitters look into the Text column for processing",
+            advanced=True,
+        ),
+        BoolInput(
+            name="text_key_validator",
+            display_name="Text Key Validator",
+            advanced=True,
+            info="If True, it will turn on the Text Key Validator and will check if the Text Key is one of the keys in the Data",
+        ),
     ]
 
     outputs = [
@@ -32,7 +43,7 @@ class CreateDataComponent(Component):
 
     def update_build_config(self, build_config: dotdict, field_value: Any, field_name: str | None = None):
         if field_name == "number_of_fields":
-            default_keys = ["code", "_type", "number_of_fields", "text_key"]
+            default_keys = ["code", "_type", "number_of_fields", "text_key", "text_key_validator"]
             try:
                 field_value_int = int(field_value)
             except ValueError:
@@ -65,6 +76,15 @@ def update_build_config(self, build_config: dotdict, field_value: Any, field_nam
         return build_config
 
     async def build_data(self) -> Data:
+        data = self.get_data()
+        return_data = Data(data=data, text_key=self.text_key)
+        self.status = return_data
+        if self.text_key_validator:
+            self.add_validator()
+        return return_data
+
+    def get_data(self):
+        """Fucntion to get the Data from the attributes"""
         data = {}
         for value_dict in self._attributes.values():
             if isinstance(value_dict, dict):
@@ -73,9 +93,13 @@ async def build_data(self) -> Data:
                     key: value.get_text() if isinstance(value, Data) else value for key, value in value_dict.items()
                 }
                 data.update(value_dict)
-        return_data = Data(data=data, text_key=self.text_key)
-        self.status = return_data
-        return return_data
+        return data
+
+    def add_validator(self):
+        """This fucntion validates that the Text Key is one of the keys in the Data"""
+        data_keys = self.get_data().keys()
+        if self.text_key not in data_keys and self.text_key != "":
+            raise ValueError(f"Text Key: {self.text_key} not found in the Data keys: {",".join(data_keys)}")
 
     def post_code_processing(self, new_frontend_node: dict, current_frontend_node: dict):
         """
diff --git a/src/backend/base/langflow/components/prototypes/UpdateData.py b/src/backend/base/langflow/components/prototypes/UpdateData.py
index 10355d2df225..e56216f24be1 100644
--- a/src/backend/base/langflow/components/prototypes/UpdateData.py
+++ b/src/backend/base/langflow/components/prototypes/UpdateData.py
@@ -1,40 +1,119 @@
-from langflow.custom import CustomComponent
+from typing import Any
+
+from langflow.custom import Component
+from langflow.inputs.inputs import IntInput, MessageTextInput, DictInput, DataInput, BoolInput
+from langflow.io import Output
+
+from langflow.field_typing.range_spec import RangeSpec
 from langflow.schema import Data
+from langflow.schema.dotdict import dotdict
 
 
-class UpdateDataComponent(CustomComponent):
-    display_name = "Update Data"
-    description = "Update Data with text-based key/value pairs, similar to updating a Python dictionary."
-    name = "UpdateData"
-
-    def build_config(self):
-        return {
-            "data": {
-                "display_name": "Data",
-                "info": "The record to update.",
-            },
-            "new_data": {
-                "display_name": "New Data",
-                "info": "The new data to update the record with.",
-                "input_types": ["Text"],
-            },
-        }
-
-    def build(
-        self,
-        data: Data,
-        new_data: dict,
-    ) -> Data:
-        """
-        Updates a record with new data.
+class UpdateOrAppendDataComponent(Component):
+    display_name: str = "Update or Append Data"
+    description: str = "Dynamically update or append data with a specified number of fields."
+    name: str = "UpdateData"
 
-        Args:
-            record (Data): The record to update.
-            new_data (dict): The new data to update the record with.
+    inputs = [
+        DataInput(
+            name="old_data",
+            display_name="Data",
+            info="The record to update.",
+        ),
+        IntInput(
+            name="number_of_fields",
+            display_name="Number of Fields",
+            info="Number of fields to be added to the record.",
+            real_time_refresh=True,
+            value=0,
+            range_spec=RangeSpec(min=1, max=15, step=1, step_type="int"),
+        ),
+        MessageTextInput(
+            name="text_key",
+            display_name="Text Key",
+            info="Key to be used as text. This is useful because more Parsers/splitters look into the Text column for processing",
+            advanced=True,
+        ),
+        BoolInput(
+            name="text_key_validator",
+            display_name="Text Key Validator",
+            advanced=True,
+            info="If True, it will turn on the Text Key Validator and will check if the Text Key is one of the keys in the Data",
+        ),
+    ]
 
-        Returns:
-            Data: The updated record.
-        """
-        data.data.update(new_data)
-        self.status = data
+    outputs = [
+        Output(display_name="Data", name="data", method="build_data"),
+    ]
+
+    def update_build_config(self, build_config: dotdict, field_value: Any, field_name: str | None = None):
+        if field_name == "number_of_fields":
+            default_keys = ["code", "_type", "number_of_fields", "text_key", "old_data", "text_key_validator"]
+            try:
+                field_value_int = int(field_value)
+            except ValueError:
+                return build_config
+            existing_fields = {}
+            if field_value_int > 15:
+                build_config["number_of_fields"]["value"] = 15
+                raise ValueError("Number of fields cannot exceed 15. Try using a Component to combine two Data.")
+            if len(build_config) > len(default_keys):
+                # back up the existing template fields
+                for key in build_config.copy():
+                    if key not in default_keys:
+                        existing_fields[key] = build_config.pop(key)
+
+            for i in range(1, field_value_int + 1):
+                key = f"field_{i}_key"
+                if key in existing_fields:
+                    field = existing_fields[key]
+                    build_config[key] = field
+                else:
+                    field = DictInput(
+                        display_name=f"Field {i}",
+                        name=key,
+                        info=f"Key for field {i}.",
+                        input_types=["Text", "Data"],
+                    )
+                    build_config[field.name] = field.to_dict()
+
+            build_config["number_of_fields"]["value"] = field_value_int
+        return build_config
+
+    async def build_data(self) -> Data:
+        new_data = self.get_data()
+        self.old_data.data.update(new_data)
+        if self.text_key:
+            self.old_data.text_key = self.text_key
+        self.status = self.old_data
+        self.add_validator(self.old_data)
+        return self.old_data
+
+    def get_data(self):
+        """Function to get the Data from the attributes"""
+        data = {}
+        for value_dict in self._attributes.values():
+            if isinstance(value_dict, dict):
+                # Check if the value of the value_dict is a Data
+                value_dict = {
+                    key: value.get_text() if isinstance(value, Data) else value for key, value in value_dict.items()
+                }
+                data.update(value_dict)
         return data
+
+    def add_validator(self, data: Data):
+        """This function validates that the Text Key is one of the keys in the Data"""
+        data_keys = data.data.keys()
+        if self.text_key not in data_keys and self.text_key != "":
+            raise ValueError(f"Text Key: {self.text_key} not found in the Data keys: {','.join(data_keys)}")
+
+    def post_code_processing(self, new_frontend_node: dict, current_frontend_node: dict):
+        """
+        This function is called after the code validation is done.
+        """
+        frontend_node = super().post_code_processing(new_frontend_node, current_frontend_node)
+        frontend_node["template"] = self.update_build_config(
+            frontend_node["template"], frontend_node["template"]["number_of_fields"]["value"], "number_of_fields"
+        )
+        frontend_node = super().post_code_processing(new_frontend_node, current_frontend_node)
+        return frontend_node
diff --git a/src/backend/base/langflow/components/prototypes/__init__.py b/src/backend/base/langflow/components/prototypes/__init__.py
index 252d1a1c5815..3a6d5d83deb5 100644
--- a/src/backend/base/langflow/components/prototypes/__init__.py
+++ b/src/backend/base/langflow/components/prototypes/__init__.py
@@ -9,7 +9,7 @@
 from .SQLExecutor import SQLExecutorComponent
 from .SubFlow import SubFlowComponent
 from .CreateData import CreateDataComponent
-from .UpdateData import UpdateDataComponent
+from .UpdateData import UpdateOrAppendDataComponent
 
 __all__ = [
     "ConditionalRouterComponent",
@@ -23,5 +23,5 @@
     "SQLExecutorComponent",
     "SubFlowComponent",
     "CreateDataComponent",
-    "UpdateDataComponent",
+    "UpdateOrAppendDataComponent",
 ]
diff --git a/src/frontend/package-lock.json b/src/frontend/package-lock.json
index 493e02b23a37..3b73d1a5bee9 100644
--- a/src/frontend/package-lock.json
+++ b/src/frontend/package-lock.json
@@ -919,6 +919,7 @@
     },
     "node_modules/@clack/prompts/node_modules/is-unicode-supported": {
       "version": "1.3.0",
+      "extraneous": true,
       "inBundle": true,
       "license": "MIT",
       "engines": {


Code review comments:
diff_hunk: @@ -23,7 +23,18 @@ class CreateDataComponent(Component):
             value=0,
             range_spec=RangeSpec(min=1, max=15, step=1, step_type="int"),
         ),
-        MessageTextInput(name="text_key", display_name="Text Key", info="Key to be used as text.", advanced=True),
+        MessageTextInput(
+            name="text_key",
+            display_name="Text Key",
+            info="Key to be used as text.This is useful cause more Parsers/ splitters look into the Text column for processing",
path: src/backend/base/langflow/components/prototypes/CreateData.py
text: @user1:
something like: info="Key that identifies the field to be used as the text content."

diff_hunk: @@ -73,9 +93,13 @@ async def build_data(self) -> Data:
                     key: value.get_text() if isinstance(value, Data) else value for key, value in value_dict.items()
                 }
                 data.update(value_dict)
-        return_data = Data(data=data, text_key=self.text_key)
-        self.status = return_data
-        return return_data
+        return data
+
+    def add_validator(self):
path: src/backend/base/langflow/components/prototypes/CreateData.py
text: @user1:
maybe `validate_text_key`

diff_hunk: @@ -1,40 +1,119 @@
-from langflow.custom import CustomComponent
+from typing import Any
+
+from langflow.custom import Component
+from langflow.inputs.inputs import IntInput, MessageTextInput, DictInput, DataInput, BoolInput
+from langflow.io import Output
+
+from langflow.field_typing.range_spec import RangeSpec
 from langflow.schema import Data
+from langflow.schema.dotdict import dotdict
 
 
-class UpdateDataComponent(CustomComponent):
-    display_name = "Update Data"
-    description = "Update Data with text-based key/value pairs, similar to updating a Python dictionary."
-    name = "UpdateData"
-
-    def build_config(self):
-        return {
-            "data": {
-                "display_name": "Data",
-                "info": "The record to update.",
-            },
-            "new_data": {
-                "display_name": "New Data",
-                "info": "The new data to update the record with.",
-                "input_types": ["Text"],
-            },
-        }
-
-    def build(
-        self,
-        data: Data,
-        new_data: dict,
-    ) -> Data:
-        """
-        Updates a record with new data.
+class UpdateOrAppendDataComponent(Component):
+    display_name: str = "Update or Append Data"
+    description: str = "Dynamically update or append data with a specified number of fields."
path: src/backend/base/langflow/components/prototypes/UpdateData.py
text: @user1:
suggest: `Dynamically update or append data with the specified fields` (just remove number)

diff_hunk: @@ -23,7 +23,18 @@ class CreateDataComponent(Component):
             value=0,
             range_spec=RangeSpec(min=1, max=15, step=1, step_type="int"),
         ),
-        MessageTextInput(name="text_key", display_name="Text Key", info="Key to be used as text.", advanced=True),
+        MessageTextInput(
+            name="text_key",
+            display_name="Text Key",
+            info="Key to be used as text.This is useful cause more Parsers/ splitters look into the Text column for processing",
+            advanced=True,
+        ),
+        BoolInput(
+            name="text_key_validator",
+            display_name="Text Key Validator",
+            advanced=True,
+            info="If True, it will turn on the Text Key Validator and will check if the Text Key is one of the keys in the Data",
path: src/backend/base/langflow/components/prototypes/CreateData.py
text: @user1:
something like: `If enabled, checks if the given "Text Key" is present in the given "Data"`

diff_hunk: @@ -1,40 +1,119 @@
-from langflow.custom import CustomComponent
+from typing import Any
+
+from langflow.custom import Component
+from langflow.inputs.inputs import IntInput, MessageTextInput, DictInput, DataInput, BoolInput
+from langflow.io import Output
+
+from langflow.field_typing.range_spec import RangeSpec
 from langflow.schema import Data
+from langflow.schema.dotdict import dotdict
 
 
-class UpdateDataComponent(CustomComponent):
-    display_name = "Update Data"
-    description = "Update Data with text-based key/value pairs, similar to updating a Python dictionary."
-    name = "UpdateData"
-
-    def build_config(self):
-        return {
-            "data": {
-                "display_name": "Data",
-                "info": "The record to update.",
-            },
-            "new_data": {
-                "display_name": "New Data",
-                "info": "The new data to update the record with.",
-                "input_types": ["Text"],
-            },
-        }
-
-    def build(
-        self,
-        data: Data,
-        new_data: dict,
-    ) -> Data:
-        """
-        Updates a record with new data.
+class UpdateOrAppendDataComponent(Component):
+    display_name: str = "Update or Append Data"
path: src/backend/base/langflow/components/prototypes/UpdateData.py
text: @user1:
update to match `UpdateData`