C# Class design for cloud storage management application
Published: 2022-05-03 13:10:41

# Summary
I want to share a C# class design pattern I recently did. I know there are a lot of different design patterns and there is no single best way, but learning different design patterns helps myself develop extensible and scalable applications.

Sample code: [blob-console-app](https://github.com/koheikawata/blob-console-app)

# TOC
- [Architecture](#architecture)
- [Class design](#class-design)
- [Codes](#codes)
  - [List file type instance](#list)
  - [for instead of foreach](#for)
  - [Dependency Injection](#dependency-injection)
  - [Upload file validation](#file-validation)
  - [Generic class](#generic-class)
  - [Metadata into dictionary](#metadata-into-dictionary)
  - [Json deserialize](#json-deserialize)

# Architecture <a name="architecture"></a>

- Operators place files which they want to upload to Azure Blob Storage.
- The .NET console app checks the pointed directory in a certain interval and if those files are valid, it uploads to the Azure Blob Storage.
- The app deletes those files from the local file system once the files are uploaded.
- The app extracts metadata and attaches the blob on cloud.
- Currently the file types are `logs` and `events` which are located different directory in the local file system.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vy6mwlvbq93tmqr6x3fl.png)

# Class design <a name="class-design"></a>

I decided to use Interface, Abstract, and sub classes because:

- Currently file types are only `Logs` and `Events`, but the system will extend. For example, `DeviceInfo` and `Requests` will be added in the future.
- It is better for future extended file types to divide the base features and ones with different requirements for each file type so the system reuses the base class features in the future.
- In order to avoid changing the base class when adding new file types, an Interface class and abstract class defines base properties, and sub classes define metadata properties.
- `FolderName` is a base property but it points to the directory for each file type. It is defined as Abstract property because Sub classes must override it and define for their own.
- `JudgeValidFile()` and `GetMetadata()` have to be implemented in sub classes, and then they are defined as Abstract. `UploadBlobAsync()`, `UploadMetadataAsync()`, and `DeleteFiles()` are defined as Abstract because they are implemented in the base class but can be overriden in sub classes accordingly.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/88zg7ib55lkh0xrwu5n6.png)

# Codes <a name="codes"></a>
## List file type instance <a name="list"></a>
- Currently the system has only two file types, `Logs` and `Events`. If you want to add another file type, only thing to do is to implement a new class for the file type and add a new instance here.

[Program.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Program.cs)
```
List<IUploadData> uploadDataList = new ()
{
    new Logs(),
    new Events(),
};
```

## for instead of foreach <a name="for"></a>

- `for (int i = 0; i < uploadDataList.Count; ++i)` processes differently depending on class instance added to the list.

- It uses `for` instead of `foreach` because the class method `GetMetadata()` gets metadata and put into its own property

[Program.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Program.cs)
```
uploadDataList[i] = uploadDataList[i].GetMetadata();
```

## Dependency Injection <a name="dependency-injection"></a>

- It does not use [dependency injection](https://docs.microsoft.com/en-us/dotnet/core/extensions/dependency-injection) but assigns the values to properties. This is because the value changes for different files found and file types. For example, `BlobClient` needs a file name and directory to instantiate for different files and different file types. It cannot create an instance in constructor.

[Program.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Program.cs)
```
List<IUploadData> uploadDataList = new()
{
    new Logs(),
    new Events(),
};

while (true)
{
    for (int i = 0; i < uploadDataList.Count; ++i)
    {
        uploadDataList[i].FileFullPaths = Directory.GetFiles(Path.Combine(localPath, uploadDataList[i].FolderName), "*", SearchOption.AllDirectories);

        foreach (string fileFullPath in uploadDataList[i].FileFullPaths)
        {
            uploadDataList[i].FileFullPath = fileFullPath;
            uploadDataList[i].FileName = Path.GetFileName(fileFullPath);
            uploadDataList[i].BlobClient = containerClient.GetBlobClient($"/{uploadDataList[i].FolderName}/{DateTime.Now.Year}/{DateTime.Now.Month}/{uploadDataList[i].FileName}");
```

## Upload file validation <a name="file-validation"></a>

- If a file found in the directory is not valid to upload, it stops the for loop and goes to a next file.
- How to valid files is different depending on file types. That is why `JudgeValidFile()` is overriden in each file type class.

[Program.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Program.cs)
```
uploadDataList[i].JudgeValidFile();
if (uploadDataList[i].IsValidFile is false) continue;
```

[Logs.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Data/Logs.cs)
```
this.IsValidFile = (Path.GetExtension(this.FileName) == ".csv" && this.FileFullPaths.Contains(this.FileFullPath + ".json"));
```

[Events.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Data/Events.cs)
```
this.IsValidFile = (Path.GetExtension(this.FileName) == ".json");
```

## Generic class <a name="generic-class"></a>

I wanted to use a generic class for `UploadMetadataAsync()` because it has to retrieve the properties dynamically depending on the sub class. I thought `UploadMetadataAsync<uploadDataList[i].GetType()>()` works but actually not. After investigating sometime, I gave up using Generic class here.

**Generic class**
```
public virtual async Task UploadMetadataAsync<T>() where T : IUploadData
{
    PropertyInfo[] propertyCollection = typeof(T).GetProperties();
    foreach (PropertyInfo property in propertyCollection)
    {
        blobMetadata[property.Name.ToString()] = property.GetValue(this).ToString();
    }
}
```

**Caller**
```
await uploadDataList[i].UploadMetadataAsync<uploadDataList[i].GetType()>().ConfigureAwait(true);
```

## Metadata into dictionary <a name="metadata-into-dictionary"></a>

- I realized I cannot use Generic class, and I was thinking how to insert only metadata properties into the dictionary for uploading. What I did is to see the difference of property between Abstract class and sub class. For example, the Abstract class `UploadData` has only the base properties `FolderName`, `IsValidFile`, `FileFullPaths`, `FileFullPath`, `FileName`, `BlobClient`, but the sub class `Logs` has the metadata properties `BeginTime`, `EndTime`, `Temperature`, `Humidity`, `Location` in addition to the base properties.

[UploadData.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Data/UploadData.cs)
```
this.blobMetadata.Clear();
PropertyInfo[] propertiesIUploadData = typeof(IUploadData).GetProperties();
IEnumerable<PropertyInfo> propertiesThis = this.GetType().GetRuntimeProperties();

foreach (PropertyInfo propertyThis in propertiesThis)
{
    int count = 0;
    foreach (PropertyInfo propertyIUploadData in propertiesIUploadData)
    {
        if (propertyThis.Name == propertyIUploadData.Name) ++count;
    }
    if (count == 0)
    {
        this.blobMetadata[propertyThis.Name.ToString()] = propertyThis.GetValue(this).ToString();
    }
}
await this.BlobClient.SetMetadataAsync(this.blobMetadata).ConfigureAwait(false);
```

## Json deserialize <a name="json-deserialize"></a>

- It is elegant to deserialize the value from a json file because I do not need to change the code `JsonSerializer.Deserialize<Logs>(jsonString)!;` for any model property. However, the problem was that the sub class has both the base property and metadata property, and when in deserialization, it sets the metadata properties with the value from a json file and also the base properties with null.
- I do not like this, but I did not come up with an elegant workaround, and what I implemented is to keep the value to local variables and then set them back after deserialization.

**json file example**
```
{
  "BeginTime": "2022-03-07T12:21:55Z",
  "EndTime": "2022-03-07T14:01:36Z",
  "Temperature": 25,
  "Humidity": 63,
  "Location": "Tokyo"
}
```

**Logs class property**
```
public string FolderName { get; }
public bool IsValidFile { get; set; }
public string[] FileFullPaths { get; set; }
public string FileFullPath { get; set; }
public string FileName { get; set; }
public BlobClient BlobClient { get; set; }
public DateTime BeginTime { get; set; }
public DateTime EndTime { get; set; }
public int Temperature { get; set; }
public int Humidity { get; set; }
public string Location { get; set; }
```

[Logs.cs](https://github.com/koheikawata/blob-console-app/blob/main/src/BlobConsoleApp/Data/Logs.cs)
```
public override IUploadData GetMetadata()
{
    bool IsValidFile = this.IsValidFile;
    string[] FileFullPaths = this.FileFullPaths;
    string FileFullPath = this.FileFullPath;
    string FileName = this.FileName;
    BlobClient BlobClient = this.BlobClient;

    string jsonString = File.ReadAllText(this.FileFullPath + ".json");
    Logs log = JsonSerializer.Deserialize<Logs>(jsonString)!;

    log.IsValidFile = IsValidFile;
    log.FileFullPaths = FileFullPaths;
    log.FileFullPath = FileFullPath;
    log.FileName = FileName;
    log.BlobClient = BlobClient;

    return log;
}
```