OSD600 - Telescope - Testing for feed URLs
Published: 2022-11-21 00:13:30

Last week, I worked on an [issue](https://github.com/Seneca-CDOT/telescope/issues/3689) for the [Seneca telescope project](https://github.com/Seneca-CDOT/telescope). The sign-up process has a section where a blog URL must be entered. This is then used to get a list of feed URLs which are used to get the posts for a user. This needed to be changed so that a feed URL could be given instead, which would then simply get shown in the feed URL list.

I looked at the service which is used to get the feed URLs from a blog URL and noticed it takes the html response of the blog URL and gets the links (`<link>` tags) by checking the `type` attribute value against a list of valid feed values. So, I decided to use a similar approach by getting the html response for a provided URL and checking the `Content-Type` header against a list of valid MIME types for a feed. I ended up updating the logic to test if a URL is a feed URL, returning it if true. If the URL is found to not be a feed URL, it would try to get the feed URLs assuming the URL is a blog URL.
I tested and confirmed that the new logic worked for both blog and feed URLs. Then, I added some tests for the new function I added to test for a feed URL. Testing this ended up being simpler than I expected as all I had to do was mock the response of a test url (using [nock](https://github.com/nock/nock)), and then check if the function returned the correct boolean value for a url.
I created a [PR](https://github.com/Seneca-CDOT/telescope/pull/3768/files) and noticed that some of the tests in another file were now failing. While I was investigating this, I got a review on my PR, requesting me to add another test to the file which had the failing tests. That file tested the API service as a whole. I found out that nock only mocks a URL's response for one request by default. And since I was now checking for a feed URL as well, the function which returned the feed URLs from a blog URL was throwing an error since the nock for that was used up. To fix this, I had to specify in the nock statement to mock the URL response for two requests:
```javascript
nock(blogUrl).get('/').twice().reply(200, mockBlogUrlResponseBody, {
  'Content-Type': 'text/html',
});
```
I fixed the failing tests and added a new test to check if trying to get the feed URLs from a feed URLs would return the feed URL back from the API service. Initially I set it up like this:
```javascript
 [
    'application/xml',
    'application/rss+xml',
    'application/atom+xml',
    'application/x.atom+xml',
    'application/x-atom+xml',
    'application/json',
    'application/json+oembed',
    'application/xml+oembed',
  ].forEach(async (type) => {
    nock(...)
    const res = await request(app)
      .post('/')
      .set('Authorization', `bearer ${createServiceToken()}`)
      .send([feedUrl]);

    expect(res.status).toBe(200);
    ...
```
However, while running the tests using [jest](https://jestjs.io/) would result in a pass, the expectations would actually fail and the failure result would show up after the tests have already ended. I was confused as to why was happening for a while. Eventually, I noticed this in the file:
```javascript
afterEach(() => {
  nock.cleanAll();
});
```
From what I understood, the test suite would run, the nocks would get cleaned before the request finished, which would be followed by the expectations. Thus, the expectations would fail after the test suite had finished. To fix this, I had to look up another way to run the same test for an array of data. I ended up using the ```javascript
it.concurrent.each([
      'application/xml',
      'application/rss+xml',
      'application/atom+xml',
      'application/x.atom+xml',
      'application/x-atom+xml',
      'application/json',
      'application/json+oembed',
      'application/xml+oembed',
])('... %s', async (type) => {
...('original test from inside the loop')
```
This ensured that the tests finished running before the test run would stop. It also allowed me to use a single test for different test values. The test statement can also be build based on the array using the %s in '... %s'.

Another thing from the review was that allowing a feed URL could break the back end as it would now store a feed URL as a blog URL. So, I investigated this and thanks to some helpful information from [alexsam29](https://github.com/alexsam29), I found that the blog URL was only stored in a html url field of the feed model. Further, I did not find any places where that field was being used. In the [posts that are shown on telescope](https://telescope.cdot.systems/), there is also a link to the blog (not the feed but the blog page). However, this link is not rendered based on the blog URL. Instead, a blog feed includes a `<link>` element which has the blog URL, and that is used to link a post to a blog:

```rss
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0">
<channel>
<title>DEV Community 👩‍💻👨‍💻: Eakam</title>
<description>The latest articles on DEV Community 👩‍💻👨‍💻 by Eakam (@eakam).</description>
<link>https://dev.to/eakam</link>
```

Therefore, I concluded that this change would not break and functionality. I tested this locally by registering two users- one with a blog URL, and one with a feed URL, and did not notice any difference. So, I added a comment about my this to the PR.