How to Load Millions of Rows from JSON Data Sources in Your .NET App
Tags: webdev, tutorial, dotnet, devops
Published: 2022-09-30 19:40:40

---
canonical_url: https://www.grapecity.com/blogs/how-to-load-millions-of-rows-from-json-data-sources-in-your-net-app
description: Learn how to load millions of rows from JSON data sources in your .NET app. See more from ComponentOne today.
---
[ComponentOne DataConnectors](https://www.grapecity.com/componentone/docs/services/online-dataconnector/overview.html "https://www.grapecity.com/componentone/docs/services/online-dataconnector/overview.html") are ADO.NET-based service components that provide a unified interface for accessing different data sources for efficient data communication. It reduces the time required to develop data-driven apps. Nowadays, the use of the JSON data source is currently becoming increasingly common to bind applications, and GrapeCity provides a [DataConnector for JSON](https://www.grapecity.com/componentone/docs/services/online-dataconnector/jsongettingstarted.html "https://www.grapecity.com/componentone/docs/services/online-dataconnector/jsongettingstarted.html") that allows easy access to JSON data using ADO.NET or EntityFramework Core. It simplifies the process of working with JSON data and gives you the ability to query both flat and relational JSON data.

Generally, if you have a huge number of records in your JSON, for example in millions, then it would take so long to assign the data by iterating through it but with the JSON DataConnector, you can load as much as data in just a few seconds and currently, this functionality is available using [Top level Document Model](https://www.grapecity.com/componentone/docs/services/online-dataconnector/jsondatamodels.html?highlight=Top%20level%20Document%20Model%2C "https://www.grapecity.com/componentone/docs/services/online-dataconnector/jsondatamodels.html?highlight=Top%20level%20Document%20Model%2C") only.

In this blog, we will learn how to load millions of records into the [FlexGrid](https://www.grapecity.com/componentone/docs/win/online-flexgrid/overview.html) very quickly using ADO.NET Provider for JSON, and the data we are taking is the Employees' data with four million records. The implementation steps are categorized as below:

1.  [Setup a New WinForms App with the required packages](#setup)
2.  [Create a Custom Data class to Map Fields with the JSON data](#create)
3.  [Bind FlexGrid to JSON Datasource](#bind)

### <a id="setup"></a>Setup a New WinForms App with the Required package

Create a new .NET 6 WinForms App using Visual Studio 2022 and add the following packages to the project:

*   [C1.Win.FlexGrid](https://www.nuget.org/packages/C1.Win.FlexGrid)
*   [C1.Win.Themes](https://www.nuget.org/packages/C1.Win.Themes)
*   [C1.AdoNet.Json](https://www.nuget.org/packages/C1.AdoNet.Json)
*   [C1.DataCollection.AdoNet](https://www.nuget.org/packages/C1.DataCollection.AdoNet)
*   [C1.DataCollection.BindingList](https://www.nuget.org/packages/C1.DataCollection.BindingList)
*   [C1.DataConnector](https://www.nuget.org/packages/C1.DataConnector)

**<u>Note:</u>** Here, we are using the [C1Themes](https://www.grapecity.com/componentone/docs/win/online-themes/overview.html) class of FlexGrid to give it an enhanced look by applying the MaterialDark theme.

Please see the following screenshots, which show the step-by-step representation of how to set up the application.

Step 1

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cd6giceqemkpsog5ttnz.png)

Step 2

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qxqbu8vw1lmp7qwv1zr3.png)

Step 3

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7tmituaisyl3rxa757fp.png)

Step 4

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/di4dxro6895vveqxgfve.png)

Step 5

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0rw5hxf06gqzm4e8z2mw.png)

Step 6

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ak9564zol7vcma3t6yi4.png)

### <a id="create"></a>Create a Custom Data class to Map Fields with the JSON data

Create a new custom data class named **Data,** implementing the [INotifyPropertyChanged](https://docs.microsoft.com/en-us/dotnet/api/system.componentmodel.inotifypropertychanged?view=net-6.0), [IEditableObject](https://docs.microsoft.com/en-us/dotnet/api/system.componentmodel.ieditableobject?view=net-6.0)Interfaces as per the schema in the JSON, which will be used by the generic class **C1AdoNetCursorDataCollection<class>** to get strongly-typed records from the JSON data source.

```
    internal class Data : INotifyPropertyChanged, IEditableObject
        {
            // Fields
            private string index;
            private string firstName;
            private string lastName;
            private bool isActive;
            private string salary;
            private string age;
            private string gender;
            private string company;
            private string email;
            private string phone;
            private string address;

            [DataMember(Name = "index")]
            public string Index
            {
                get => index;
                set
                {
                    if (index != value)
                    {
                        index = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "firstName")]
            public string FirstName
            {
                get => firstName;
                set
                {
                    if (firstName != value)
                    {
                        firstName = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "lastname")]
            public string Lastname
            {
                get => lastName;
                set
                {
                    if (lastName != value)
                    {
                        lastName = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "salary")]
            public string Salary
            {
                get => salary;
                set
                {
                    if (salary != value)
                    {
                        salary = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "age")]
            public string Age
            {
                get => age;
                set
                {
                    if (age != value)
                    {
                        age = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "gender")]
            public string Gender
            {
                get => gender;
                set
                {
                    if (gender != value)
                    {
                        gender = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "company")]
            public string Company
            {
                get => company;
                set
                {
                    if (company != value)
                    {
                        company = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "email")]
            public string Email
            {
                get => email;
                set
                {
                    if (email != value)
                    {
                        email = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "phone")]
            public string Phone
            {
                get => phone;
                set
                {
                    if (phone != value)
                    {
                        phone = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "address")]
            public string Address
            {
                get => address;
                set
                {
                    if (address != value)
                    {
                        address = value;
                        OnPropertyChanged();
                    }
                }
            }

            [DataMember(Name = "isActive")]
            public bool IsActive
            {
                get => isActive;
                set
                {
                    if (isActive != value)
                    {
                        isActive = value;
                        OnPropertyChanged();
                    }
                }
            }

            // INotifyPropertyChanged Members
            public event PropertyChangedEventHandler PropertyChanged;

            private void OnPropertyChanged([CallerMemberName] string propertyName = "")
            {
                OnPropertyChanged(new PropertyChangedEventArgs(propertyName));
            }

            protected void OnPropertyChanged(PropertyChangedEventArgs e)
            {
                PropertyChanged?.Invoke(this, e);

            }

            // IEditableObject Members
            private Data _clone;

            public void BeginEdit()
            {
                _clone = (Data)MemberwiseClone();
            }

            public void CancelEdit()
            {
                if (_clone != null)
                {
                    foreach (var p in GetType().GetRuntimeProperties())
                    {
                        if (p.CanRead && p.CanWrite)
                        {
                            p.SetValue(this, p.GetValue(_clone, null), null);
                        }
                    }
                }
            }
            public void EndEdit()
            {
                _clone = null;
            }
        }

```

### <a id="bind"></a>Bind FlexGrid to JSON Datasource

In the final step, you need to build a connection with your JSON data, as suggested [here](https://www.grapecity.com/componentone/docs/services/online-dataconnector/jsonconnection.html), and wrap your **CursorDataCollection** object in **C1DataCollectionBindingList** to bind it to an instance of FlexGrid.

```
    C1AdoNetCursorDataCollection<Data> dataCollection;
    string documentConnectionString = $"Data Model=Document;Uri=" + @"..\..\..\4mData.json" + ";Json Path='$.employees';Max Page Size=1000";
    var conn = new C1JsonConnection(documentConnectionString);
    dataCollection = new C1AdoNetCursorDataCollection<Data>(conn, "employees");
    await dataCollection.LoadMoreItemsAsync();
```

Here we are using the LoadOnDemand feature to load the data, so you need to handle the [AfterScroll](https://www.grapecity.com/componentone/docs/win/online-flexgrid/C1.Win.C1FlexGrid.4.5.2~C1.Win.C1FlexGrid.C1FlexGrid~AfterScroll_EV.html?highlight=afterscroll%2C) event of FlexGrid as given below to load the data asynchronously as soon as the user scrolls to the last visible row.

```
    c1FlexGrid1.AfterScroll += C1FlexGrid1_AfterScroll;
    private void C1FlexGrid1_AfterScroll(object sender, C1.Win.FlexGrid.RangeEventArgs e)
    {
    if (e.NewRange.BottomRow == c1FlexGrid1.Rows.Count - 1)
    {
    _ = dataCollection.LoadMoreItemsAsync();
    }
    }
```

After implementing all the above-given steps, the final application works as shown in the GIF below:

![Scroll](https://files.grapecity.com/gc-website-media/q0tgstuy/scroll.gif?width=500&height=254.18060200668896)

Hopefully, you will enjoy this demo of creating a large data application using the ADO.NET Provider for JSON. You can download the sample from here.

Download the [sample](https://global-cdn.grapecity.com/blogs/componentone/20220922-how-to-load-millions-of-rows-from-json-data-sources-in-your-net-app/Load_4M_JSON_Data_FlexGrid_Demo.zip) for this blog.

Please feel free to try it out and leave your feedback or questions in the comments section. Happy Coding!