Transform Your Data Import Process
Tags: database, import, ai, datascience
Published: 2023-04-13 11:19:08

_Importing data is one of the first meaningful interactions your customer will have with your product. Let’s explore how to make it as seamless as possible._


Data onboarding generally is the starting point for interactions between B2B software applications and their customers. It is the starting point of interaction between your application and customers. Studies show that 86% of customers are willing to pay a higher price for a better UX, which can increase customer satisfaction.
‍

External data often comes in files such as Excel, CSV, and JSON that lack a clear organization, making it challenging to work with. In fact, data cleaning and mapping can take up to 80% of a data scientist’s time, leaving little room for analysis and insight. This problem is not limited to small businesses; even large companies struggle with the data onboarding process due to the sheer volume of data. Furthermore, the high and recurring manual effort for internal onboarding and engineering teams can pose severe bottlenecks in onboarding new customers in time.


Data onboarding is challenging due to the lack of standardization in data formats, even within a single file type like CSV. This makes it difficult to create automated processes to clean and map data. Additionally, data quality issues such as incomplete, inconsistent, and inaccurate data can lead to flawed and faulty conclusions, affecting decision-making, particularly for those companies that rely on data for compliance purposes.


To address these challenges, many companies are trying to implement a scalable approach that allows automating the data cleaning and mapping processes using machine learning. nuvo Data Importer is one of the solutions that can help companies streamline their data onboarding processes. It offers a range of cleaning functions that help automate the data cleaning and mapping process by reducing the time and resources required for data onboarding.


The nuvo Data Importer’s ML algorithms and flexible cleaning functions can nearly cover every use case by transforming data and displaying customized messages to the user during the import process. Similarly, companies can reduce the time and resources required for data onboarding by almost 90% using the nuvo Data Importer.


## What exactly are cleaning functions?


Specific import workflow events trigger cleaning functions and serve as callback functions. Users can receive feedback and automatically transform imported data to meet the required format using these functions. This enables them to achieve faster and higher-quality data submissions without requiring direct user interaction.


For example, cleaning functions can compare retrieved and imported data, identify duplicate entries, display errors, enrich entries, establish dependencies between columns, merge or split multiple columns, call third-party APIs for verification purposes, and many more tasks.


### Review Entries Step


nuvo offers different types of cleaning functions to provide high flexibility in validating and automatically reformatting data. Now we will take a closer look at two functions: onEntryInit and columnHook.


### onEntryInit


This function enables iteration through all imported entries/rows by providing access to every value within each iteration. Users can use this function to establish column-cross dependencies between two or more columns, such as merge, split, or value-based Regular Expressions (Regex) checks. In the example below, the validation of countries is contingent upon the continent selected.


```
     onEntryInit={(row, index) => {
     if (row.continent) {
        const matchFound = data.find((set)=> set.country === row.country);
         // check if selected country belongs to the continent
        if (matchFound?.continent !== row.continent) {
           return {
             country: {
               info: [
                 {
                   message: "Country does not belong to " + row.continent,
                   level: "warning"
                 }
               ]
             }
           };
        }
    }
 }}
```
 

[CodeSandbox Link
](https://b1u7n8.csb.app/)
‍

### columnHook


All the data in a particular column can be accessed inside the columnHook function. This enables you to apply custom data transformation and provides feedback to the user within the UI in the form of custom info, warnings, and error messages. The example below shows how columnHooks can conduct server callbacks to validate the incoming data with the backend and check for duplicate entries. In the case of a duplicate, a custom error message is shown, allowing the user to edit the record prior to importing it.
 

```
 columnHooks={{
     email: async (values) => {
       let registeredEmails;
       await fetch(
         "https://my-json-server.typicode.com/comdocks/nuvo/customers"
       )
         .then((response) => response.json())
         .then((json) => {
           registeredEmails = json.map((row) => row.email);
         });
   
       const duplicateErrors = [];
   
       values.forEach((entry) => {
         //warn user if email already exists
         if (registeredEmails.includes(entry.value)) {
           duplicateErrors.push([
             {
               info: [
                 {
                   message:
                     "Duplicate entry. The email address already exists.",
                   level: "error"
                 }
               ]
             },
             entry.index
           ]);
         }
       });
   
       return duplicateErrors;
     }
 }}
```
 

[CodeSandbox Link
](https://ys1278.csb.app/)
‍

The onEntryInit and columnHook functions provide high flexibility in validating and automatically reformatting data, making data onboarding faster and more efficient. Companies can greatly benefit from using nuvo Data Importer, which enables better decision-making and increases customer satisfaction.


If you’re looking to streamline your data onboarding process and optimize your data-driven decision-making, try out nuvo Data Importer. With its powerful cleaning functions and machine learning algorithms, it can cover nearly every use case and significantly reduce the time and resources required for data onboarding.

‍
Want to see a live demo or receive a free trial? [Book a call](https://www.getnuvo.com/contact-us) with the team for a quick demo.

‍
Source: UserTesting. (2019). The ROI of CX: Why you can’t afford to ignore customer experience. Retrieved from https://www.usertesting.com/blog/the-roi-of-cx-why-you-cant-afford-to-ignore-customer-experience/