Release 04 part3: Changing the lexer algorithm
Published: 2020-12-11 16:04:16

### Read the previous blog please (this blog is written assuming readers already read the previous blog)

### Before start
Previosuly, I used es6 arrow function, but after I went through the program, I realized that the program did not use arrow function at all, so I changed the new arrow function to the es5 function. 
```
const regexTerms = () => {
function regexTerms() {
...

const resolveRegex = () => {
function resolveRegex() {
...
```

Also, most of the code was using `let`, even if the variable was actually constant. So I changed them either.
```
  let rawRegex = regexTerms.join("|");
  let mainRegex = new RegExp(rawRegex, "gi");
  let mainRegex = regexTerms();
  let resolverRegexes = resolveRegex();

```
```
  const rawRegex = regexTerms.join("|");
  const mainRegex = new RegExp(rawRegex, "gi");
  const mainRegex = regexTerms();
  const resolverRegexes = resolveRegex(lexerItem);
```
### Problem
So, the main problem was there was too many loop to generate "lexer object" to be compiled after all. 

Simply the algorithm looks like this,
```
// three levels loop
lexerResults.map(lexerResult=>{
   for(let i = 0; i < resolverRegexes.length; i++){
      ...
      Object.keys(tokens.tokens).forEach((token)=>{
       ...
      })
   }
})
```
So, inside the `map` loop, there is `for` loop, then inside the `for` there is also `foreach` loop. If we can solve this problem out. (For example, reduce `loop`), the efficiency will be enhanced.

### Solution
So, essentially `resolveLexerItemMetadata` was doing compare first, then make the object later (which was made in another function beforehand). But, what if the process is combined into one single function, I mean, compare and make the object at the same time.
```
// function: creating the object.
function resolveRegex() {
  let resolverRegexes = [];
  Object.keys(tokens.tokens).forEach((token) => {
    resolverRegexes.push({
      match: new RegExp(`(${tokens.tokens[token].match})`, "gi"),
      data: {
        identifier: token,
      },
    });

// function: comparing with the given token and object
unction resolveLexerItemMetadata(lexerItem) {
  let data = {};
  const resolverRegexes = resolveRegex(lexerItem);
  for (let i = 0; i < resolverRegexes.length; i++) {
    if (lexerItem.match(resolverRegexes[i].match) != null) {
      data["identifier"] = resolverRegexes[i].data.identifier;
      data["token"] = lexerItem;

      break;
    }
  }
  return data;
}
```

### Solution 2
```
/* Instead of creating an array of objects for Token to be compared with given lexerItm,
*  Find the token with the lexerItm, then return a single object
*  Previous version, let n = the num of tokens, the best case is Ω(n) runtime.
*  This version, let n = the num of tokens, the best case is Ω(1) runtime.
*/
function resolveRegex(lexerItem) {
  const foundToken = Object.keys(tokens.tokens).find((token) => {
    const matchReg = new RegExp(`(${tokens.tokens[token].match})`, "gi");
    return lexerItem.match(matchReg) != null ? true : false;
  });
```
Instead of creating a `resolvedToken` object (intermediate object), simply find the token and if found, creating a final version of object.

```
function parseLine(line) {
  const lexerResults = line.match(mainRegex); // The Regex result is quickly translated into an array
  let lexeredLine = [];
  lexerResults.map((lexerResult) => {
    lexeredLine.push(resolveRegex(lexerResult));
  });
  return lexeredLine;

```
By doing so, we can remove the `resolveLexerItemMetadata` function and put the process altogether.
This will results in reducing one extra loop (`foreach` loop).

After testing the runtime, I could see the runtime is much faster: approximately 20%. 

Visit the [PR](https://github.com/PrivateGER/GreenLight/pull/21/commits/930c229932a6600a83494e41345de422344084ab)

### Conclusion
At first, I tried to contribute to the bigger project (even if a small change). However, It is hard to find a good one for me even though I watched lecture and  tried to practice. 
The simple bug is already taken by others, and lots of code I cannot understand. 
The reason, I think, is I might be too afraid of failing to create PR. So personally, I score myself 80/100

Even though I could not achieve my goal(contributing to the real project like react.js, next.js ...), I learned a lot of Open Source Development community and environment. How to look for contribution, license, unit testing, git command and communicate with open source development community. 

These assets are not something to disappear. This semester is to be end, but my programming experience just begin. I will keep practice the thing I have learned in this course and try to keep connecting with the community. And later I will be able to contribute to the real world project.