Fun with speech detection using RxJS and Angular standalone components
Tags: angular, rxjs, typescript
Published: 2023-03-16 03:22:56

##Introduction
This is day 20 of Wes Bos's JavaScript 30 challenge where I build a speech detection application using RxJS, Angular standalone components and Web Speech API. Angular not only can call it’s own APIs to render components but it can also interact with Web Speech API to guess what I spoke in English and outputted its confidence level. The API amazed me due to its high accuracy and confidence level as if a real human being was listening to me and understood my accent.

###Create a new Angular project
```bash
ng generate application day20-speech-detection-standalone
```

###Bootstrap AppComponent
First, I convert AppComponent into standalone component such that I can bootstrap AppComponent and inject providers in main.ts.

```typescript
// app.component.ts

import { Component } from '@angular/core';
import { Title } from '@angular/platform-browser';
import { SpeechDetectionComponent } from './speech-detection/speech-detection/speech-detection.component';

@Component({
  selector: 'app-root',
  standalone: true,
  imports: [SpeechDetectionComponent],
  template: '<app-speech-detection></app-speech-detection>',
  styles: [
    `
      :host {
        display: block;
      }
    `,
  ],
})
export class AppComponent {
  title = 'Day20 Speech Detection Standalone';

  constructor(titleService: Title) {
    titleService.setTitle(this.title);
  }
}
```

In Component decorator, I put `standalone: true` to convert AppComponent into a standalone component.

Instead of importing `SpeechDetectionComponent` in `AppModule`, I import `SpeechDetectionComponent` (that is also a standalone component) in the imports array because the inline template references it. It is because main.ts uses `bootstrapApplication` to render `AppComponent` as the root component of the application. When compiler sees &lt;app-speech-detection&gt; in the inline template and `AppComponent` does not import `SpeechDetectionComponent`, the application fails to compile.

```typescript
// main.ts

import { bootstrapApplication } from '@angular/platform-browser';
import { AppComponent } from './app/app.component';

bootstrapApplication(AppComponent).catch((err) => console.error(err));
```

Next, I can delete AppModule that has no use now.

###Declare speech detection component
I declare standalone component, `SpeechDetectionComponent`, to start the speech detection process. To verify the component is standalone, `standalone: true` is specified in the Component decorator.

```bash
src/app/
├── app.component.spec.ts
├── app.component.ts
└── speech-detection
    ├── helpers
    │   └── speech-detection.helper.ts
    ├── interfaces
    │   └── speech-recognition.interface.ts
    └── speech-detection
        ├── speech-detection.component.spec.ts
        └── speech-detection.component.ts
```

The application use this component to do everything and the tag is &lt;app-speech-detection&gt;.

```typescript
// speech-detection.component.ts

import { AsyncPipe, NgFor, NgIf } from '@angular/common';
import { ChangeDetectionStrategy, Component, OnDestroy, OnInit } from '@angular/core';
import {
  createRecognition,
  createRecognitionSubscription,
  createWordListObservable,
} from '../helpers/speech-detection.helper';

@Component({
  selector: 'app-speech-detection',
  standalone: true,
  imports: [AsyncPipe, NgFor, NgIf],
  template: ` <div class="words" contenteditable>
    <ng-container *ngIf="wordList$ | async as wordList">
      <p *ngFor="let word of wordList">{{ word.transcript }}, confidence: {{ word.confidencePercentage }}%</p>
    </ng-container>
  </div>`,
  styles: [`...omitted due to bervity...`],
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class SpeechDetectionComponent implements OnInit, OnDestroy {
  recognition = createRecognition();
  subscription = createRecognitionSubscription(this.recognition);
  wordList$ = createWordListObservable(this.recognition);

  ngOnInit(): void {
    this.recognition.start();
  }

  ngOnDestroy(): void {
    this.subscription.unsubscribe();
  }
}
```

`SpeechDetectionComponent` imports `AsyncPipe`, `NgFor` and `NgIf` in the imports array because the inline template makes use of `ngIf`, `ngFor` and `async`.

Add speech detection helper to define observable and subscription
I moved speech detection observable and subscription to helper file to maintain small component file and good project structure.

```typescript
// speech-recognition.interface
export interface SpeechRecognitionInfo {
  transcript: string;
  confidence: number;
  isFinal: boolean;
}

export type Transcript = Omit<SpeechRecognitionInfo, 'isFinal' | 'confidence'> & { confidencePercentage: string };
// speech-detection.helper.ts

import { fromEvent, tap, map, filter, scan } from 'rxjs';
import {
  SpeechRecognitionInfo,
  Transcript,
} from '../interfaces/speech-recognition.interface';

declare var webkitSpeechRecognition: any;
declare var SpeechRecognition: any;

export const createRecognition = () => {
  const recognition = new webkitSpeechRecognition() || new SpeechRecognition();
  recognition.interimResults = true;
  recognition.lang = 'en-US';

  return recognition;
};

export const createRecognitionSubscription = (recognition: any) =>
  fromEvent(recognition, 'end')
    .pipe(tap(() => recognition.start()))
    .subscribe();

export const createWordListObservable = (recognition: any) => {
  const percent = 100;
  return fromEvent(recognition, 'result').pipe(
    map((e: any): SpeechRecognitionInfo => {
      const transcript = Array.from(e.results)
        .map((result: any) => result[0].transcript)
        .join('');
      const poopScript = transcript.replace(/poop|poo|shit|dump/gi, '💩');
      const firstResult = e.results[0];

      return {
        transcript: poopScript,
        confidence: firstResult[0].confidence,
        isFinal: firstResult.isFinal,
      };
    }),
    filter(({ isFinal }) => isFinal),
    scan(
      (acc: Transcript[], { transcript, confidence }) =>
        acc.concat({
          transcript,
          confidencePercentage: (confidence * percent).toFixed(2),
        }),
      [],
    ),
  );
};
```

####Explain createRecognition

`createRecognition` function instantiates a `SpeechRecognition` service, sets the language to English and interimResult to true.

####Explain createRecognitionSubscription

`createRecognitionSubscription` function restarts speech recognition and subscribes the Observable after the previous speech recognition ends.

- fromEvent(recognition, 'end') – Emit value when end event of speech recognition service occurs
- tap(() => recognition.start()) – restart speech recognition to recognize the next sentence
- subscribe() – subscribe observable

####Explain createWordListObservable

`createWordListObservable` function emits a list of phrases recognized by Web Speech API.

- fromEvent(recognition, 'result') – Speech recognition emits a word or phrase
- map() – replace foul languages with emoji and return the phrase, confidential level and the isFinal flag
- filter(({ isFinal }) => isFinal) – emit value when the final result is ready
- scan() – append the phrase and confidence level to the word list

####Demystifies RxJS logic in SpeechDetectionComponent

`SpeechDetectionComponent` class is succinct now after refactoring Observable and Subscription codes to `speech-detection.helper.ts`. They are extracted to functions in helper file

```typescript
recognition = createRecognition();
subscription = createRecognitionSubscription(this.recognition);
wordList$ = createWordListObservable(this.recognition);
The component declares this.recognition that is a SpeechRecognition service.
```

`this.wordList$` is an Observable that renders a list of words or phrases.

```typescript
<ng-container *ngIf="wordList$ | async as wordList">
    <p *ngFor="let word of wordList">
       {{ word.transcript }}, confidence: {{ word.confidencePercentage }}%
    </p>
</ng-container>
```

`this.subscription` is a subscription that unsubscribe in ngOnDestroy

```typescript
ngOnDestroy(): void {
  this.subscription.unsubscribe();
}
```

This is it and I have built a speech recognition application with RxJS, Angular standalone components and Web Speech API.

##Final Thoughts
In this post, I show how to use RxJS, Angular standalone components and web speech API to build speech detection. The application has the following characteristics after using Angular 15’s new features:

- The application does not have NgModules and constructor boilerplate codes.
- The standalone component is very clean because I moved as many RxJS codes to separate helper file as possible.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in Angular and other technologies.

##Resources:
- Github Repo: https://github.com/railsstudent/ng-rxjs-30/tree/main/projects/day20-speech-detection-standalone
- Live demo: https://railsstudent.github.io/ng-rxjs-30/day20-speech-detection-standalone/
- Wes Bos's JavaScript 30 Challenge: https://github.com/wesbos/JavaScript30