News Archiver
Tags: appwritehack, appwrite, javascript
Published: 2022-04-22 14:02:21

[Instructions]: # (To submit to the Appwrite Hackathon on DEV, please fill out all sections.)


### Overview of My Submission
Archives headlines/content from news websites (selected news website which are for now New18 and IndiaToday it can be change) every 3hr. which allows users to historically see the news and how it’s reported on different sites.
By using a package node-schedule to run background job every 3hr. and **puppeteer** for scraping the content from a website and this date will be saved in **Appwrite** database.
Render this data on a client-side application (website).


### Submission Category: 
Web2 Wizards

### Link to Code
**Frontend/Client-Side Application** 
{% embed https://github.com/AnujSsStw/newsarchive-appwrite %} 
[Live website](https://forappwrite--effulgent-florentine-2fb423.netlify.app/) without data because I didn't host my application on a appwrite droplet to collect the data and display it sorry for that 🙇‍♀️.

![Web app](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0msgvmzmb8ylni4usfpg.PNG)
<u></u>
**Backend/Server-Side Application** 
{% embed https://github.com/AnujSsStw/appwrite-backend %}

![Code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6pnms8wzt4pvhp6do2aa.PNG)

### Additional Resources / Info
Do check it out in 1.5x for a quick walkthrough on my application and how it works
{% embed https://youtu.be/1TiwWn6y-Mk %}

### A quick walk through of my Application and basically how it works

PS- I didn't focus on security so that's why my id is still shows in main app. And if I have to I can just set it the .env

**Backend**

- Run node index.js

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f7ueupl3d4l8ky6cbccc.PNG)
- Cron schedule can be set to any time (for now let say every min * * * * *)
- After that it Scrape data mainly img and headline

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ajd3hemgtfru7nscns6s.PNG)
- Now coming up it check the collection list
   - Basically for 1 things
   - Does a collection exit with current date it if not then create a collection with given attributes 
   - If collection exits then create the document in that collection with the scraped data

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q8htmq76qbdiqwy41xhs.PNG)

1.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3uzn8lhqhdx9dqnfkxc3.png)

2.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/999pwvovdgxgmtsair1y.png)

- That's it for the Backend
**Fontend**

- Renders the data that was in collection

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/byum1mb53a61ea12pgyv.png)


`And how it render the data of the selected date?
Well it's easy To start of I created the collection with Date id`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nmixmdn7bwrl0j9utgqc.PNG)

For more info you can connect with me on [Twitter](https://twitter.com/anujzzZtw)

### OoO
The backend can be hosted on digitalocean so it can always keep running with appwrite droplet. If you are wondering.