Final Project - Hardware Recommendation
Tags: octograd2020, githubsdp, botframework, mlnet
Published: 2020-05-21 13:45:49

> This document requires proofreading in English, thank you!

## Hardware Recommendation

I am a programmer, but for the family and friends I also help with computer repair. Over the time I collected more and more used hardware components such as Memory sticks and HDDs, and when I took care of a friend or family laptop, I also added it a used component (if it was suitable).

Also, as the Hardware Champion, I was also asked to recommend which new computer to buy.

So I had two problems:

 1. As the number of the used hardware grows, how can I know if I have a component to upgrade a specific laptop?
 2. Can I trust my suggestions for new computers, if I don't track on the market updates, and if I don't know what are the Designer user needs?

These two questions lead me to try to implement a recommendation system for computer upgrading or buying.

> **Spoiler:** I didn't finish the system, but if someone wants to build one, I will be happy to join!

I started to define the project with the *Lean Canvas Model*, I'm really suggesting you take this pattern seriously. We all know how many ideas and startup didn't even start, and not talking about finishing because we think our idea is huge, but we don't have actually measurement tools to validate that. So please use the *Lean Canvas Model* to stay focused and adjust your project in any step!

The main problems that the project will solve are:

 1. Peoples want to upgrade their computer or buy a new one, but they didn't know **which** hardware component **effect** on which parameter!  
  For example, someone can think that the new processor will help the computer to run faster, but he has an HDD disk, so the computer will still to be slowly.

 2. Peoples want to buy a computer for their needs, but they didn't know **what their needs are!**  
  For example, a student wants to buy a computer for learning Architecture. He knows which software he will be going to use, but he won't know if he needs a good processor or a lot of memory.

In the *Lean Canvas Model* there are more three steps, and the central one is the **Unique Value Proposition**, but I will skip on them to the **Solution**:

1. Collect Hardware Information from users and give them an upgrade recommendation, collect the Hardware Information to your recommendation system.
2. Create an informative survey and generate an answer by calculating the user requirement, user personal details, and output from the recommendation system.

> Note that even the main goal of the 1st is to collect the data, you have to give a value to the user, otherwise, he will not give you any data.

## Link to Code

[Landing Page](https://baruchiro.github.io/HWRecommendation-WebAPI/)

{% github https://github.com/baruchiro/HWRecommendation-WebAPI no-readme %}

{% github https://github.com/baruchiro/HWRecommendation-local no-readme %}


## How I built it

As I said, I didn't complete the system, so there is no complete software architecture map here, so you've earned it, otherwise, it was boring.

In this project I took the opportunity to try two new technologies from Microsoft. `C#` is my native language, and I actually love the new Microsoft (I think anyone who insists on staying prejudiced loses).

The two new technologies are *Microsoft Bot Framework* and *ML.NET* ("New", as of the end of 2018)

> I've learned and dealt with a lot of things since then, so I can't remember all the details. 
During the development period, I wanted to write an article about my experiences with these new systems, but in those days I had hard writing in English. (Even now my English is not good, but at least it's not hard for me to write :-))

### Microsoft Bot Framework

This system felt good. In those days, I started to love the idea of API, the idea of writing a system to expose the data and action, and then write any type of interfaces (CLI, web app, bots) to interact with the API.

The *Bot Framework* is based on *ASP.NET*, so it is just another controller that you can add to your existing *ASP.NET Web API* (At the time it seemed right to me. Now I understand that it has no meaning).

The *Bot Framework* wants to be an endpoint from all the bot systems (Telegram, Facebook, etc), and it was such a comfortable system. It has also an "Emulator", you can try your bot easily with a local tool, so it was excellent.

Unfortunately, I missed the opportunity to offer improvements. I solved some problems and invented 2-3 patterns to reuse code and solve some other annoying design problems, but as I said, in those days it was harder for me to speak English.

At least I've contributed some code for testing:

{% github https://github.com/microsoft/botbuilder-dotnet/pull/1922 %}

> Edit: Funny thing! I didn't remember if and where I wrote my conclusions from the *Bot Framework* experience, but when I edited this *DEV.to* profile I found a draft about that! 
I don't know if it is still relevant, if so, I will love to get help posting it. 
{% link https://dev.to/baruchiro/bot-framework-net-design-decisions-5gl1 %}


### ML.NET

In contrast to the *Bot Framework* with the potential to improve, the *ML.NET* was very disappointing.

I played with ML with `python` before, and compared to the dynamization in `python`, `C#` was disturbed. What is one of the big changes between `python` and `C#`? Right! the **Typing System**.

In ML, before you doing your magic, or even before you give the algorithm or the library to do its magic, you have to transform and clean your data. This is the real work on ML, at least from my point of view as a developer who has just experimented.

If you writing in `python`, do any transform you want, and get the transformed data when you want to, the transformed data will be different type from the input (for example, you transformed strings to numbers), and `python` don't do a type validation if it not have to, so print your data and see it state every step you want!

But in *ML.NET*... Oh, it was horrible!

First of all, I built a whole system to flatten complex objects and convert then to a very short list of types that can be input to the *ML.NET*, I think it was in [this file](https://github.com/baruchiro/HWRecommendation-WebAPI/blob/master/RecommendationAlgorithms/AlgorithmManager/Extensions/TypeExtensions.cs).

Then, I started to transform my data. There was a good pattern of "pipeline", you connect one pipeline to another to stream the information from one transformation to another. **But you always have to be aware of the type of the current state of the data!**. If I remember correctly, the pipelines connected each other with a kind of limitation about the expected output and input type.

Even if I don't remember correctly, for sure you want to see how the data looks like after some transformations, to validate you are on the right way. So gues what? **You need a type!**

Yes! If you want to see how the data looks like after all your transformation, before the real algorithm transformation, you have to represent the expected output with Class. You find yourself trying to do all the transformations in your head, in order to represent the expected output in a Class, all this to get a snapshot of the transformed data, after you did all this in your head, and absolutely missed some edge cases.

This is the stage that broke me.

And my issue is still opened:

{% github https://github.com/dotnet/machinelearning/issues/3829 %}

And these are [my other issues](https://github.com/dotnet/machinelearning/issues?q=author%3Abaruchiro+).


## Additional Thoughts / Feelings / Stories

### Github Student Pack

I used *Github Sdutent Pack* to easily freely use *Azure*. In the *Microsoft* ecosystem, you can easily create an end-to-end development environment, so I developed in my local *Visual Studio*, then, from the *Visual Studio* I configured the CI/CD pipelines in *Azure DevOps*, and from there, I published to *Azure* without testing my time in configuring a cloud systems.

So when my class friends had to display everything on their laptops, include demo and let peoples to all wait for their laptop to be available, I just had to give them QR Code to experiment on their own.
In other words, **I was in production**.

### Today

Now, as I write this article, I go back to that period (just a year ago), and I realize that I would have done things completely differently.

In the last few months I learned so much new things and architectures, so there are some things I would have done otherwise.

#### Doing one thing

In Agile world, we need to always give a value. Sure, you need some infrastructure to start giving a value, but the thout about my project is that I had a hidden propose in this project. I wanted to index all my hardware components.

Remember my goals at the beginning of the post? Is that part of the goal?

No.

If you trying to do multiple things at once, you disperse and you have nothing left.

#### Microservices

From above:

> So it is just another controller that you can add to your existing *ASP.NET Web API*

If you creating a one project for all your needs, you can't change things easly.

After realizing that *ML.NET* can't give me what I need, I thought to use `python`, but the cost of integrating the `python` code to my project was expensive. (I still used `python` for the preparation transformations).

If you working with *Microservices*, you have a contract and way to communicate between the services, and since each service need to do one thing, if the service not meet your needs, you can easily implement another one, in another technology, that will replace the existed.

Each service is small **and independent**, so it's cheap to replace it.