How to Run Python Script as a Service (Windows & Linux)
Tags: webdev, tutorial, productivity, python
Published: 2023-03-31 08:34:16


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t7qpk6yh96at4rameeyz.png)<p class="c18"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 624.00px; height: 312.00px;"><img alt="" src="images/image6.png" style="width: 624.00px; height: 312.00px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span></p><p class="c9"><span class="c2 c12 c15">In this dynamic environment, gathering time-sensitive public data only once is useless as it quickly becomes obsolete. To be competitive, you must keep your data fresh and run your web scraping scripts repeatedly and regularly.</span></p><p class="c9"><span class="c2 c12 c15">The easiest way is to run a script in the background. In other words, run it as a service. Fortunately, no matter the operating system in use &ndash; Linux or Windows &ndash; you have great tools at your disposal. This guide will detail the process in a few simple steps. </span></p><h2 class="c28" id="h.nfatgfldgid2"><span class="c26 c11">Preparing a Python script for Linux</span></h2><p class="c19"><span class="c12 c15">In this article, information from </span><span class="c24 c12"><a class="c22" href="https://www.google.com/url?q=http://books.toscrape.com/&amp;sa=D&amp;source=editors&amp;ust=1680250635900223&amp;usg=AOvVaw34-WhiH63KN5Uv69GCQVNC">a list of book URLs</a></span><span class="c2 c12 c15">&nbsp;will be scraped. When the process reaches the end of the list, it loops over and refreshes the data again and again.</span></p><p class="c19"><span class="c12 c15">First, make a request and retrieve the HTML content of a page. Use the </span><span class="c12 c32"><a class="c22" href="https://www.google.com/url?q=https://pypi.org/project/requests/&amp;sa=D&amp;source=editors&amp;ust=1680250635900518&amp;usg=AOvVaw3HHwPPuxIHrRB_-yVc153y">Requests module</a></span><span class="c2 c12 c15">&nbsp;to do so:</span></p><a id="t.a2eecb675e511568dc2bd6ca03fc8d9d775277ed"></a><a id="t.0"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">urls </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">[</span></p><p class="c5"><span class="c3">&#39;https://books.toscrape.com/catalogue/sapiens-a-brief-history-of-humankind_996/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c3">&#39;https://books.toscrape.com/catalogue/shakespeares-sonnets_989/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c3">&#39;https://books.toscrape.com/catalogue/sharp-objects_997/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c1">]</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">index </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c10">0</span></p><p class="c5"><span class="c4">while</span><span class="c0">&nbsp;</span><span class="c10">True</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; url </span><span class="c6">=</span><span class="c0">&nbsp;urls</span><span class="c1">[</span><span class="c0">index </span><span class="c6">%</span><span class="c0">&nbsp;</span><span class="c8">len</span><span class="c1">(</span><span class="c0">urls</span><span class="c1">)]</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; index </span><span class="c6">+=</span><span class="c0">&nbsp;</span><span class="c10">1</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c8">print</span><span class="c1">(</span><span class="c3">&#39;Scraping url&#39;</span><span class="c1">,</span><span class="c0">&nbsp;url</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; response </span><span class="c6">=</span><span class="c0">&nbsp;requests</span><span class="c1">.</span><span class="c8">get</span><span class="c1">(</span><span class="c0">url</span><span class="c1">)</span></p></td></tr></table><p class="c13"><span class="c12">Once the content is retrieved, parse it using the </span><span class="c24 c12"><a class="c22" href="https://www.google.com/url?q=https://beautiful-soup-4.readthedocs.io/en/latest/&amp;sa=D&amp;source=editors&amp;ust=1680250635904873&amp;usg=AOvVaw3fxVXcgTQx68h4I_uJo36Z">Beautiful Soup</a></span><span class="c2 c11">&nbsp;library:</span></p><a id="t.cfe36bf3466df4168e71850985298f9aeae507fc"></a><a id="t.1"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">soup </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">BeautifulSoup</span><span class="c1">(</span><span class="c0">response</span><span class="c1">.</span><span class="c0">content</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;html.parser&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">book_name </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select_one</span><span class="c1">(</span><span class="c3">&#39;.product_main&#39;</span><span class="c1">).</span><span class="c0">h1</span><span class="c1">.</span><span class="c2 c0">text</span></p><p class="c5"><span class="c0">rows </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select</span><span class="c1">(</span><span class="c3">&#39;.table.table-striped tr&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">product_info </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">{</span><span class="c0">row</span><span class="c1">.</span><span class="c0">th</span><span class="c1">.</span><span class="c0">text</span><span class="c6">:</span><span class="c0">&nbsp;row</span><span class="c1">.</span><span class="c0">td</span><span class="c1">.</span><span class="c0">text </span><span class="c4">for</span><span class="c0">&nbsp;row </span><span class="c4">in</span><span class="c0">&nbsp;rows</span><span class="c1">}</span></p></td></tr></table><p class="c13"><span class="c2 c11">Make sure your data directory-to-be already exists, and then save book information there in JSON format.</span></p><p class="c18"><span class="c31">Protip:</span><span class="c12">&nbsp;use the </span><span class="c24 c12"><a class="c22" href="https://www.google.com/url?q=https://docs.python.org/3/library/pathlib.html&amp;sa=D&amp;source=editors&amp;ust=1680250635907758&amp;usg=AOvVaw0te0Dm3Xermr0InJ0APQb0">pathlib module</a></span><span class="c2 c11">&nbsp;to automatically convert Python path separators into a format compatible with both Windows and Linux systems.</span></p><p class="c18 c16"><span class="c2 c11"></span></p><a id="t.9e613f9ded7612e7ce7cbce34dce11b2a1cfef58"></a><a id="t.2"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">data_folder </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">Path</span><span class="c1">(</span><span class="c3">&#39;./data&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">data_folder</span><span class="c1">.</span><span class="c8">mkdir</span><span class="c1">(</span><span class="c0">parents</span><span class="c6">=</span><span class="c0">True</span><span class="c1">,</span><span class="c0">&nbsp;exist_ok</span><span class="c6">=</span><span class="c0">True</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">json_file_name </span><span class="c6">=</span><span class="c0">&nbsp;re</span><span class="c1">.</span><span class="c8">sub</span><span class="c1">(</span><span class="c3">&#39;[: ]&#39;</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;-&#39;</span><span class="c1">,</span><span class="c0">&nbsp;book_name</span><span class="c1">)</span></p><p class="c5"><span class="c0">json_file_path </span><span class="c6">=</span><span class="c0">&nbsp;data_folder </span><span class="c6">/</span><span class="c0">&nbsp;f</span><span class="c3">&#39;{json_file_name}.json&#39;</span></p><p class="c5"><span class="c4">with</span><span class="c0">&nbsp;</span><span class="c8">open</span><span class="c1">(</span><span class="c0">json_file_path</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;w&#39;</span><span class="c1">)</span><span class="c0">&nbsp;</span><span class="c4">as</span><span class="c0">&nbsp;</span><span class="c10">book_file</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; json</span><span class="c1">.</span><span class="c8">dump</span><span class="c1">(</span><span class="c0">product_info</span><span class="c1">,</span><span class="c0">&nbsp;book_file</span><span class="c1">)</span></p></td></tr></table><p class="c17"><span class="c12">Since this script is long-running and never exits, you must also handle any requests from the operating system attempting to shut down the script. This way, you can finish the current iteration before exiting. To do so, you can define a class that handles the </span><span class="c24 c12"><a class="c22" href="https://www.google.com/url?q=https://docs.python.org/3/library/signal.html&amp;sa=D&amp;source=editors&amp;ust=1680250635911589&amp;usg=AOvVaw0bMPP1UbQhcoltsi2qsz-Z">operating system signals</a></span><span class="c2 c11">:</span></p><p class="c17 c16"><span class="c2 c11"></span></p><a id="t.0f3ed5006f046da07215d0943e9fc1cb61079364"></a><a id="t.3"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c4">class</span><span class="c0">&nbsp;</span><span class="c8">SignalHandler</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; shutdown_requested </span><span class="c6">=</span><span class="c2 c0">&nbsp;False</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">__init__</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; signal</span><span class="c1">.</span><span class="c8">signal</span><span class="c1">(</span><span class="c0">signal</span><span class="c1">.</span><span class="c10">SIGINT</span><span class="c1">,</span><span class="c0">&nbsp;self</span><span class="c1">.</span><span class="c0">request_shutdown</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; signal</span><span class="c1">.</span><span class="c8">signal</span><span class="c1">(</span><span class="c0">signal</span><span class="c1">.</span><span class="c10">SIGTERM</span><span class="c1">,</span><span class="c0">&nbsp;self</span><span class="c1">.</span><span class="c0">request_shutdown</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">request_shutdown</span><span class="c1">(</span><span class="c0">self</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c6">*</span><span class="c0">args</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c8">print</span><span class="c1">(</span><span class="c3">&#39;Request to shutdown received, stopping&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; self</span><span class="c1">.</span><span class="c0">shutdown_requested </span><span class="c6">=</span><span class="c2 c0">&nbsp;True</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">can_run</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c4">return</span><span class="c0">&nbsp;not self</span><span class="c1">.</span><span class="c0">shutdown_requested</span></p></td></tr></table><p class="c13"><span class="c12">Instead of having a loop condition that never changes (</span><span class="c14">while True</span><span class="c12">), you can ask the newly built </span><span class="c14">SignalHandler</span><span class="c2 c11">&nbsp;whether any shutdown signals have been received:</span></p><a id="t.70c0c7d586a3cd3ca10ea2dc75d9b4752a96d752"></a><a id="t.4"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">signal_handler </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">SignalHandler</span><span class="c1">()</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0"># </span><span class="c6">...</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">while</span><span class="c0">&nbsp;signal_handler</span><span class="c1">.</span><span class="c8">can_run</span><span class="c1">()</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; # run the code only </span><span class="c4">if</span><span class="c0">&nbsp;you don&#39;t need to exit</span></p></td></tr></table><p class="c13"><span class="c2 c11">Here&rsquo;s the code so far:</span></p><a id="t.fb08b917f98331e0f4f3313625be2f2ceac5208a"></a><a id="t.5"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;json</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;re</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;signal</span></p><p class="c5"><span class="c0">from pathlib </span><span class="c4">import</span><span class="c2 c0">&nbsp;Path</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;requests</span></p><p class="c5"><span class="c0">from bs4 </span><span class="c4">import</span><span class="c2 c0">&nbsp;BeautifulSoup</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">class</span><span class="c0">&nbsp;</span><span class="c8">SignalHandler</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; shutdown_requested </span><span class="c6">=</span><span class="c2 c0">&nbsp;False</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">__init__</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; signal</span><span class="c1">.</span><span class="c8">signal</span><span class="c1">(</span><span class="c0">signal</span><span class="c1">.</span><span class="c10">SIGINT</span><span class="c1">,</span><span class="c0">&nbsp;self</span><span class="c1">.</span><span class="c0">request_shutdown</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; signal</span><span class="c1">.</span><span class="c8">signal</span><span class="c1">(</span><span class="c0">signal</span><span class="c1">.</span><span class="c10">SIGTERM</span><span class="c1">,</span><span class="c0">&nbsp;self</span><span class="c1">.</span><span class="c0">request_shutdown</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">request_shutdown</span><span class="c1">(</span><span class="c0">self</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c6">*</span><span class="c0">args</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c8">print</span><span class="c1">(</span><span class="c3">&#39;Request to shutdown received, stopping&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; self</span><span class="c1">.</span><span class="c0">shutdown_requested </span><span class="c6">=</span><span class="c2 c0">&nbsp;True</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">can_run</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c4">return</span><span class="c0">&nbsp;not self</span><span class="c1">.</span><span class="c2 c0">shutdown_requested</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">signal_handler </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">SignalHandler</span><span class="c1">()</span></p><p class="c5"><span class="c0">urls </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">[</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c3">&#39;https://books.toscrape.com/catalogue/sapiens-a-brief-history-of-humankind_996/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c3">&#39;https://books.toscrape.com/catalogue/shakespeares-sonnets_989/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c3">&#39;https://books.toscrape.com/catalogue/sharp-objects_997/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c1">]</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">index </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c10">0</span></p><p class="c5"><span class="c4">while</span><span class="c0">&nbsp;signal_handler</span><span class="c1">.</span><span class="c8">can_run</span><span class="c1">()</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; url </span><span class="c6">=</span><span class="c0">&nbsp;urls</span><span class="c1">[</span><span class="c0">index </span><span class="c6">%</span><span class="c0">&nbsp;</span><span class="c8">len</span><span class="c1">(</span><span class="c0">urls</span><span class="c1">)]</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; index </span><span class="c6">+=</span><span class="c0">&nbsp;</span><span class="c10">1</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c8">print</span><span class="c1">(</span><span class="c3">&#39;Scraping url&#39;</span><span class="c1">,</span><span class="c0">&nbsp;url</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; response </span><span class="c6">=</span><span class="c0">&nbsp;requests</span><span class="c1">.</span><span class="c8">get</span><span class="c1">(</span><span class="c0">url</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; soup </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">BeautifulSoup</span><span class="c1">(</span><span class="c0">response</span><span class="c1">.</span><span class="c0">content</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;html.parser&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; book_name </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select_one</span><span class="c1">(</span><span class="c3">&#39;.product_main&#39;</span><span class="c1">).</span><span class="c0">h1</span><span class="c1">.</span><span class="c2 c0">text</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; rows </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select</span><span class="c1">(</span><span class="c3">&#39;.table.table-striped tr&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; product_info </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">{</span><span class="c0">row</span><span class="c1">.</span><span class="c0">th</span><span class="c1">.</span><span class="c0">text</span><span class="c6">:</span><span class="c0">&nbsp;row</span><span class="c1">.</span><span class="c0">td</span><span class="c1">.</span><span class="c0">text </span><span class="c4">for</span><span class="c0">&nbsp;row </span><span class="c4">in</span><span class="c0">&nbsp;rows</span><span class="c1">}</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; data_folder </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">Path</span><span class="c1">(</span><span class="c3">&#39;./data&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; data_folder</span><span class="c1">.</span><span class="c8">mkdir</span><span class="c1">(</span><span class="c0">parents</span><span class="c6">=</span><span class="c0">True</span><span class="c1">,</span><span class="c0">&nbsp;exist_ok</span><span class="c6">=</span><span class="c0">True</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; json_file_name </span><span class="c6">=</span><span class="c0">&nbsp;re</span><span class="c1">.</span><span class="c8">sub</span><span class="c1">(</span><span class="c3">&#39;[\&#39;: ]&#39;</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;-&#39;</span><span class="c1">,</span><span class="c0">&nbsp;book_name</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; json_file_path </span><span class="c6">=</span><span class="c0">&nbsp;data_folder </span><span class="c6">/</span><span class="c0">&nbsp;f</span><span class="c3">&#39;{json_file_name}.json&#39;</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c4">with</span><span class="c0">&nbsp;</span><span class="c8">open</span><span class="c1">(</span><span class="c0">json_file_path</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;w&#39;</span><span class="c1">)</span><span class="c0">&nbsp;</span><span class="c4">as</span><span class="c0">&nbsp;</span><span class="c10">book_file</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; json</span><span class="c1">.</span><span class="c8">dump</span><span class="c1">(</span><span class="c0">product_info</span><span class="c1">,</span><span class="c0">&nbsp;book_file</span><span class="c1">)</span></p></td></tr></table><p class="c17"><span class="c2 c11">The script will refresh JSON files with newly collected book information.</span></p><h2 class="c28" id="h.9xigexe4xych"><span class="c26 c11">Running a Linux daemon</span></h2><p class="c25"><span class="c2 c11">If you&rsquo;re wondering how to run a Python script in Linux, there are multiple ways to do it on startup. Many distributions have built-in GUI tools for such purposes.</span></p><p class="c25"><span class="c12">Let&rsquo;s use one of the most popular distributions, </span><span class="c24 c12"><a class="c22" href="https://www.google.com/url?q=https://linuxmint.com/download.php&amp;sa=D&amp;source=editors&amp;ust=1680250635933135&amp;usg=AOvVaw2FkNMLvcmA_Hl7wnzIod1R">Linux Mint</a></span><span class="c2 c11">, as an example. It uses a desktop environment called Cinnamon that provides a startup application utility.</span></p>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gm32wcdxrnnrmfktxqkr.png)<p class="c18"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 624.00px; height: 486.67px;"><img alt="" src="images/image4.png" style="width: 624.00px; height: 486.67px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span></p><p class="c13"><span class="c2 c11">It allows you to add your script and specify a startup delay.</span></p> 
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2c9w33xtt74s5sgds0dt.png)<p class="c17"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 624.00px; height: 485.33px;"><img alt="" src="images/image5.png" style="width: 624.00px; height: 485.33px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span></p><p class="c17"><span class="c2 c11">However, this approach doesn&rsquo;t provide more control over the script. For example, what happens when you need to restart it?</span></p><p class="c17"><span class="c12">This is where </span><span class="c14">systemd</span><span class="c2 c11">&nbsp;comes in. Systemd is a service manager that allows you to manage user processes using easy-to-read configuration files.</span></p><p class="c13"><span class="c12">To use </span><span class="c14">systemd</span><span class="c12">, let&rsquo;s first create a file in the </span><span class="c14">/etc/systemd/system</span><span class="c2 c11">&nbsp;directory:</span></p><a id="t.1d960ee576465a0cdca964376dbaa0da5ca912f8"></a><a id="t.6"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">cd </span><span class="c6">/</span><span class="c0">etc</span><span class="c6">/</span><span class="c0">systemd</span><span class="c6">/</span><span class="c2 c0">system</span></p><p class="c5"><span class="c0">touch book</span><span class="c6">-</span><span class="c0">scraper</span><span class="c1">.</span><span class="c0">service</span></p></td></tr></table><p class="c13"><span class="c12">Add the following content to the </span><span class="c14">book-scraper.service</span><span class="c2 c11">&nbsp;file using your favorite editor:</span></p><a id="t.300a3d9b4a88c77501be71fa9760dfcf7c2ae143"></a><a id="t.7"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c1">[</span><span class="c0">Unit</span><span class="c1">]</span></p><p class="c5"><span class="c0">Description</span><span class="c6">=</span><span class="c10">A</span><span class="c0">&nbsp;script </span><span class="c4">for</span><span class="c2 c0">&nbsp;scraping the book information</span></p><p class="c5"><span class="c0">After</span><span class="c6">=</span><span class="c0">syslog</span><span class="c1">.</span><span class="c0">target network</span><span class="c1">.</span><span class="c2 c0">target</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c1">[</span><span class="c0">Service</span><span class="c1">]</span></p><p class="c5"><span class="c0">WorkingDirectory</span><span class="c6">=/</span><span class="c0">home</span><span class="c6">/</span><span class="c0">oxylabs</span><span class="c6">/</span><span class="c2 c0">Scraper</span></p><p class="c5"><span class="c0">ExecStart</span><span class="c6">=/</span><span class="c0">home</span><span class="c6">/</span><span class="c0">oxylabs</span><span class="c6">/</span><span class="c0">Scraper</span><span class="c6">/</span><span class="c0">venv</span><span class="c6">/</span><span class="c0">bin</span><span class="c6">/</span><span class="c0">python3 scrape</span><span class="c1">.</span><span class="c2 c0">py</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">Restart</span><span class="c6">=</span><span class="c2 c0">always</span></p><p class="c5"><span class="c0">RestartSec</span><span class="c6">=</span><span class="c10">120</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c1">[</span><span class="c0">Install</span><span class="c1">]</span></p><p class="c5"><span class="c0">WantedBy</span><span class="c6">=</span><span class="c0">multi</span><span class="c6">-</span><span class="c0">user</span><span class="c1">.</span><span class="c0">target</span></p></td></tr></table><p class="c17"><span class="c2 c11">Here&rsquo;s the basic rundown of the parameters used in the configuration file:</span></p><ul class="c35 lst-kix_5x3jbvn6fzgz-0 start"><li class="c17 c29 li-bullet-0"><span class="c14">After</span><span class="c2 c11">&nbsp;&ndash; ensures you only start your Python script once the network is up. </span></li><li class="c17 c29 li-bullet-0"><span class="c14">RestartSec</span><span class="c2 c11">&nbsp;&ndash; sleep time before restarting the service.</span></li><li class="c17 c29 li-bullet-0"><span class="c14">Restart</span><span class="c2 c11">&nbsp;&ndash; describes what to do if a service exits, is killed, or a timeout is reached. </span></li><li class="c17 c29 li-bullet-0"><span class="c14">WorkingDirectory</span><span class="c2 c11">&nbsp;&ndash; current working directory of the script.</span></li><li class="c17 c29 li-bullet-0"><span class="c14">ExecStart</span><span class="c2 c11">&nbsp;&ndash; the command to execute.</span></li></ul><p class="c13"><span class="c12">Now, it&rsquo;s time to tell </span><span class="c14">systemd</span><span class="c12">&nbsp;about the newly created daemon. Run the </span><span class="c14">daemon-reload</span><span class="c2 c11">&nbsp;command:</span></p><a id="t.2701e52e98f4e20fbc206caf24154ff7a4b8aba1"></a><a id="t.8"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">systemctl</span><span class="c0">&nbsp;daemon</span><span class="c6">-</span><span class="c0">reload</span></p></td></tr></table><p class="c13"><span class="c2 c11">Then, start your service:</span></p><a id="t.4dbcd17aaa09a761fa4d60b0caf63617e4f9ab75"></a><a id="t.9"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c30">systemctl</span><span class="c30">&nbsp;start book</span><span class="c30">-</span><span class="c30">scraper</span></p></td></tr></table><p class="c13"><span class="c2 c11">And finally, check whether your service is running:</span></p><a id="t.82ec35a4f4ab935293f5e3f50bda429a568e2463"></a><a id="t.10"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">$ </span><span class="c0">systemctl</span><span class="c0">&nbsp;status book</span><span class="c6">-</span><span class="c2 c0">scraper</span></p><p class="c5"><span class="c0">book</span><span class="c6">-</span><span class="c0">scraper</span><span class="c1">.</span><span class="c0">service </span><span class="c6">-</span><span class="c0">&nbsp;</span><span class="c10">A</span><span class="c0">&nbsp;script </span><span class="c4">for</span><span class="c2 c0">&nbsp;scraping the book information</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp;</span><span class="c10">Loaded</span><span class="c6">:</span><span class="c0">&nbsp;</span><span class="c8">loaded</span><span class="c0">&nbsp;</span><span class="c1">(</span><span class="c6">/</span><span class="c0">etc</span><span class="c6">/</span><span class="c0">systemd</span><span class="c6">/</span><span class="c0">system</span><span class="c6">/</span><span class="c0">book</span><span class="c6">-</span><span class="c0">scraper</span><span class="c1">.</span><span class="c0">service</span><span class="c1">;</span><span class="c0">&nbsp;disabled</span><span class="c1">;</span><span class="c0">&nbsp;vendor preset</span><span class="c6">:</span><span class="c0">&nbsp;enabled</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp;</span><span class="c10">Active</span><span class="c6">:</span><span class="c0">&nbsp;</span><span class="c8">active</span><span class="c0">&nbsp;</span><span class="c1">(</span><span class="c0">running</span><span class="c1">)</span><span class="c0">&nbsp;since Thu </span><span class="c10">2022</span><span class="c6">-</span><span class="c10">09</span><span class="c6">-</span><span class="c10">08</span><span class="c0">&nbsp;</span><span class="c10">15</span><span class="c6">:</span><span class="c10">01</span><span class="c6">:</span><span class="c10">27</span><span class="c0">&nbsp;</span><span class="c10">EEST</span><span class="c1">;</span><span class="c2 c0">&nbsp;16min ago</span></p><p class="c5"><span class="c0">&nbsp; &nbsp;Main </span><span class="c10">PID</span><span class="c6">:</span><span class="c0">&nbsp;</span><span class="c10">60803</span><span class="c0">&nbsp;</span><span class="c1">(</span><span class="c0">python3</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; </span><span class="c10">Tasks</span><span class="c6">:</span><span class="c0">&nbsp;</span><span class="c10">1</span><span class="c0">&nbsp;</span><span class="c1">(</span><span class="c0">limit</span><span class="c6">:</span><span class="c0">&nbsp;</span><span class="c10">18637</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp;</span><span class="c10">Memory</span><span class="c6">:</span><span class="c0">&nbsp;</span><span class="c10">21</span><span class="c1">.</span><span class="c2 c0">3M</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp;</span><span class="c10">CGroup</span><span class="c6">:</span><span class="c0">&nbsp;</span><span class="c6">/</span><span class="c0">system</span><span class="c1">.</span><span class="c0">slice</span><span class="c6">/</span><span class="c0">book</span><span class="c6">-</span><span class="c0">scraper</span><span class="c1">.</span><span class="c2 c0">service</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;</span><span class="c10">60803</span><span class="c0">&nbsp;</span><span class="c6">/</span><span class="c0">home</span><span class="c6">/</span><span class="c0">oxylabs</span><span class="c6">/</span><span class="c0">Scraper</span><span class="c6">/</span><span class="c0">venv</span><span class="c6">/</span><span class="c0">bin</span><span class="c6">/</span><span class="c0">python3 scrape</span><span class="c1">.</span><span class="c2 c0">py</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">Sep </span><span class="c10">08</span><span class="c0">&nbsp;</span><span class="c10">15</span><span class="c6">:</span><span class="c10">17</span><span class="c6">:</span><span class="c10">55</span><span class="c0">&nbsp;laptop python3</span><span class="c1">[</span><span class="c10">60803</span><span class="c1">]</span><span class="c6">:</span><span class="c0">&nbsp;Scraping url https</span><span class="c6">://</span><span class="c0">books</span><span class="c1">.</span><span class="c0">toscrape</span><span class="c1">.</span><span class="c0">com</span><span class="c6">/</span><span class="c0">catalogue</span><span class="c6">/</span><span class="c0">shakespeares</span><span class="c6">-</span><span class="c0">sonnets_989</span><span class="c6">/</span><span class="c0">index</span><span class="c1">.</span><span class="c2 c0">html</span></p><p class="c5"><span class="c0">Sep </span><span class="c10">08</span><span class="c0">&nbsp;</span><span class="c10">15</span><span class="c6">:</span><span class="c10">17</span><span class="c6">:</span><span class="c10">55</span><span class="c0">&nbsp;laptop python3</span><span class="c1">[</span><span class="c10">60803</span><span class="c1">]</span><span class="c6">:</span><span class="c0">&nbsp;Scraping url https</span><span class="c6">://</span><span class="c0">books</span><span class="c1">.</span><span class="c0">toscrape</span><span class="c1">.</span><span class="c0">com</span><span class="c6">/</span><span class="c0">catalogue</span><span class="c6">/</span><span class="c0">sharp</span><span class="c6">-</span><span class="c0">objects_997</span><span class="c6">/</span><span class="c0">index</span><span class="c1">.</span><span class="c2 c0">html</span></p><p class="c5"><span class="c0">Sep </span><span class="c10">08</span><span class="c0">&nbsp;</span><span class="c10">15</span><span class="c6">:</span><span class="c10">17</span><span class="c6">:</span><span class="c10">55</span><span class="c0">&nbsp;laptop python3</span><span class="c1">[</span><span class="c10">60803</span><span class="c1">]</span><span class="c6">:</span><span class="c0">&nbsp;Scraping url https</span><span class="c6">://</span><span class="c0">books</span><span class="c1">.</span><span class="c0">toscrape</span><span class="c1">.</span><span class="c0">com</span><span class="c6">/</span><span class="c0">catalogue</span><span class="c6">/</span><span class="c0">sapiens</span><span class="c6">-</span><span class="c0">a</span><span class="c6">-</span><span class="c0">brief</span><span class="c6">-</span><span class="c0">history</span><span class="c6">-</span><span class="c4">of</span><span class="c6">-</span><span class="c0">humankind_996</span><span class="c6">/</span><span class="c0">index</span><span class="c1">.</span><span class="c0">html</span></p></td></tr></table><p class="c17"><span class="c31">Protip</span><span class="c12">: use </span><span class="c14">journalctl</span><span class="c14">&nbsp;-S today -u book-scraper.service</span><span class="c2 c11">&nbsp;to monitor your logs in real-time.</span></p><p class="c17"><span class="c12">Congrats! Now you can control your service via </span><span class="c14">systemd</span><span class="c2 c11">.</span></p><h2 class="c28" id="h.qg1561ikgce7"><span class="c26 c11">Running a Python script as a Windows service</span></h2><p class="c25"><span class="c2 c11">Running a Python script as a Windows service is not as straightforward as one might expect. Let&rsquo;s start with the script changes.</span></p><p class="c25"><span class="c2 c11">To begin, change how the script is executed based on the number of arguments it receives from the command line. &nbsp;</span></p><p class="c18"><span class="c12">If the script receives a single argument, assume that Windows Service Manager is attempting to start it. It means that you have to run an initialization code. If zero arguments are passed, print some helpful information by using </span><span class="c14">win32serviceutil.HandleCommandLine</span><span class="c2 c11">:</span></p><p class="c18 c16"><span class="c2 c11"></span></p><a id="t.0c20fcef547e9ea0a0b2ddd2d2a6fdfa938b968c"></a><a id="t.11"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c4">if</span><span class="c0">&nbsp;__name__ </span><span class="c6">==</span><span class="c0">&nbsp;</span><span class="c3">&#39;__main__&#39;</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c4">if</span><span class="c0">&nbsp;</span><span class="c8">len</span><span class="c1">(</span><span class="c0">sys</span><span class="c1">.</span><span class="c0">argv</span><span class="c1">)</span><span class="c0">&nbsp;</span><span class="c6">==</span><span class="c0">&nbsp;</span><span class="c10">1</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; servicemanager</span><span class="c1">.</span><span class="c8">Initialize</span><span class="c1">()</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; servicemanager</span><span class="c1">.</span><span class="c8">PrepareToHostSingle</span><span class="c1">(</span><span class="c0">BookScraperService</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; servicemanager</span><span class="c1">.</span><span class="c8">StartServiceCtrlDispatcher</span><span class="c1">()</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c4">else</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; win32serviceutil</span><span class="c1">.</span><span class="c8">HandleCommandLine</span><span class="c1">(</span><span class="c0">BookScraperService</span><span class="c1">)</span></p></td></tr></table><p class="c13"><span class="c12">Next, extend the special utility class and set some properties. The service name, display name, and description will all be visible in the Windows services utility (</span><span class="c14">services.msc</span><span class="c2 c11">) once your service is up and running.</span></p><a id="t.d5dc09c33d7211709c3052b627ef9cbe8ae5c8e3"></a><a id="t.12"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c4">class</span><span class="c0">&nbsp;</span><span class="c8">BookScraperService</span><span class="c1">(</span><span class="c0">win32serviceutil</span><span class="c1">.</span><span class="c0">ServiceFramework</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; _svc_name_ </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c3">&#39;BookScraperService&#39;</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; _svc_display_name_ </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c3">&#39;BookScraperService&#39;</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; _svc_description_ </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c3">&#39;Constantly updates the info about books&#39;</span></p></td></tr></table><p class="c17"><span class="c12">Finally, implement the </span><span class="c14">SvcDoRun</span><span class="c12">&nbsp;and </span><span class="c14">SvcStop</span><span class="c2 c11">&nbsp;methods to start and stop the service. Here&rsquo;s the script so far:</span></p><p class="c17 c16"><span class="c2 c11"></span></p><a id="t.09878645bcd39bd889797299dbee1ad771fa41a5"></a><a id="t.13"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;sys</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;servicemanager</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;win32event</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;win32service</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;win32serviceutil</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;json</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;re</span></p><p class="c5"><span class="c0">from pathlib </span><span class="c4">import</span><span class="c2 c0">&nbsp;Path</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;requests</span></p><p class="c5"><span class="c0">from bs4 </span><span class="c4">import</span><span class="c2 c0">&nbsp;BeautifulSoup</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">class</span><span class="c0">&nbsp;</span><span class="c8">BookScraperService</span><span class="c1">(</span><span class="c0">win32serviceutil</span><span class="c1">.</span><span class="c0">ServiceFramework</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; _svc_name_ </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c3">&#39;BookScraperService&#39;</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; _svc_display_name_ </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c3">&#39;BookScraperService&#39;</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; _svc_description_ </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c3">&#39;Constantly updates the info about books&#39;</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">__init__</span><span class="c1">(</span><span class="c0">self</span><span class="c1">,</span><span class="c0">&nbsp;args</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; win32serviceutil</span><span class="c1">.</span><span class="c0">ServiceFramework</span><span class="c1">.</span><span class="c8">__init__</span><span class="c1">(</span><span class="c0">self</span><span class="c1">,</span><span class="c0">&nbsp;args</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; self</span><span class="c1">.</span><span class="c0">event </span><span class="c6">=</span><span class="c0">&nbsp;win32event</span><span class="c1">.</span><span class="c8">CreateEvent</span><span class="c1">(</span><span class="c0">None</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c10">0</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c10">0</span><span class="c1">,</span><span class="c0">&nbsp;None</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">GetAcceptedControls</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; result </span><span class="c6">=</span><span class="c0">&nbsp;win32serviceutil</span><span class="c1">.</span><span class="c0">ServiceFramework</span><span class="c1">.</span><span class="c8">GetAcceptedControls</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; result </span><span class="c6">|=</span><span class="c0">&nbsp;win32service</span><span class="c1">.</span><span class="c10">SERVICE_ACCEPT_PRESHUTDOWN</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c4">return</span><span class="c2 c0">&nbsp;result</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">SvcDoRun</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; urls </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">[</span></p><p class="c5"><span class="c3">&#39;https://books.toscrape.com/catalogue/sapiens-a-brief-history-of-humankind_996/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c3">&#39;https://books.toscrape.com/catalogue/shakespeares-sonnets_989/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c3">&#39;https://books.toscrape.com/catalogue/sharp-objects_997/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c1">]</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; index </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c10">0</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c4">while</span><span class="c0">&nbsp;</span><span class="c10">True</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; result </span><span class="c6">=</span><span class="c0">&nbsp;win32event</span><span class="c1">.</span><span class="c8">WaitForSingleObject</span><span class="c1">(</span><span class="c0">self</span><span class="c1">.</span><span class="c0">event</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c10">5000</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c4">if</span><span class="c0">&nbsp;result </span><span class="c6">==</span><span class="c0">&nbsp;win32event</span><span class="c1">.</span><span class="c10">WAIT_OBJECT_0</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c4">break</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; url </span><span class="c6">=</span><span class="c0">&nbsp;urls</span><span class="c1">[</span><span class="c0">index </span><span class="c6">%</span><span class="c0">&nbsp;</span><span class="c8">len</span><span class="c1">(</span><span class="c0">urls</span><span class="c1">)]</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; index </span><span class="c6">+=</span><span class="c0">&nbsp;</span><span class="c10">1</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c8">print</span><span class="c1">(</span><span class="c3">&#39;Scraping url&#39;</span><span class="c1">,</span><span class="c0">&nbsp;url</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; response </span><span class="c6">=</span><span class="c0">&nbsp;requests</span><span class="c1">.</span><span class="c8">get</span><span class="c1">(</span><span class="c0">url</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; soup </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">BeautifulSoup</span><span class="c1">(</span><span class="c0">response</span><span class="c1">.</span><span class="c0">content</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;html.parser&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; book_name </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select_one</span><span class="c1">(</span><span class="c3">&#39;.product_main&#39;</span><span class="c1">).</span><span class="c0">h1</span><span class="c1">.</span><span class="c2 c0">text</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; rows </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select</span><span class="c1">(</span><span class="c3">&#39;.table.table-striped tr&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; product_info </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">{</span><span class="c0">row</span><span class="c1">.</span><span class="c0">th</span><span class="c1">.</span><span class="c0">text</span><span class="c6">:</span><span class="c0">&nbsp;row</span><span class="c1">.</span><span class="c0">td</span><span class="c1">.</span><span class="c0">text </span><span class="c4">for</span><span class="c0">&nbsp;row </span><span class="c4">in</span><span class="c0">&nbsp;rows</span><span class="c1">}</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; data_folder </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">Path</span><span class="c1">(</span><span class="c3">&#39;C:\\Users\\User\\Scraper\\dist\\scrape\\data&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; data_folder</span><span class="c1">.</span><span class="c8">mkdir</span><span class="c1">(</span><span class="c0">parents</span><span class="c6">=</span><span class="c0">True</span><span class="c1">,</span><span class="c0">&nbsp;exist_ok</span><span class="c6">=</span><span class="c0">True</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; json_file_name </span><span class="c6">=</span><span class="c0">&nbsp;re</span><span class="c1">.</span><span class="c8">sub</span><span class="c1">(</span><span class="c3">&#39;[\&#39;: ]&#39;</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;-&#39;</span><span class="c1">,</span><span class="c0">&nbsp;book_name</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; json_file_path </span><span class="c6">=</span><span class="c0">&nbsp;data_folder </span><span class="c6">/</span><span class="c0">&nbsp;f</span><span class="c3">&#39;{json_file_name}.json&#39;</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c4">with</span><span class="c0">&nbsp;</span><span class="c8">open</span><span class="c1">(</span><span class="c0">json_file_path</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;w&#39;</span><span class="c1">)</span><span class="c0">&nbsp;</span><span class="c4">as</span><span class="c0">&nbsp;</span><span class="c10">book_file</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; json</span><span class="c1">.</span><span class="c8">dump</span><span class="c1">(</span><span class="c0">product_info</span><span class="c1">,</span><span class="c0">&nbsp;book_file</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; def </span><span class="c8">SvcStop</span><span class="c1">(</span><span class="c0">self</span><span class="c1">)</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; self</span><span class="c1">.</span><span class="c8">ReportServiceStatus</span><span class="c1">(</span><span class="c0">win32service</span><span class="c1">.</span><span class="c10">SERVICE_STOP_PENDING</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; win32event</span><span class="c1">.</span><span class="c8">SetEvent</span><span class="c1">(</span><span class="c0">self</span><span class="c1">.</span><span class="c0">event</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">if</span><span class="c0">&nbsp;__name__ </span><span class="c6">==</span><span class="c0">&nbsp;</span><span class="c3">&#39;__main__&#39;</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c4">if</span><span class="c0">&nbsp;</span><span class="c8">len</span><span class="c1">(</span><span class="c0">sys</span><span class="c1">.</span><span class="c0">argv</span><span class="c1">)</span><span class="c0">&nbsp;</span><span class="c6">==</span><span class="c0">&nbsp;</span><span class="c10">1</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; servicemanager</span><span class="c1">.</span><span class="c8">Initialize</span><span class="c1">()</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; servicemanager</span><span class="c1">.</span><span class="c8">PrepareToHostSingle</span><span class="c1">(</span><span class="c0">BookScraperService</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; servicemanager</span><span class="c1">.</span><span class="c8">StartServiceCtrlDispatcher</span><span class="c1">()</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c4">else</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; win32serviceutil</span><span class="c1">.</span><span class="c8">HandleCommandLine</span><span class="c1">(</span><span class="c0">BookScraperService</span><span class="c1">)</span></p></td></tr></table><p class="c17"><span class="c2 c11">Now that you have the script, open a Windows terminal of your preference.</span></p><p class="c13"><span class="c31">Protip:</span><span class="c12">&nbsp;if you&rsquo;re using Powershell, make sure to include a </span><span class="c14">.exe</span><span class="c2 c11">&nbsp;extension when running binaries to avoid unexpected errors.</span></p>

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i82r7qr1bhpm06p22373.png)<p class="c17"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 624.00px; height: 589.33px;"><img alt="" src="images/image1.png" style="width: 624.00px; height: 589.33px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span></p><p class="c13"><span class="c2 c11">Once the terminal is open, change the directory to the location of your script with a virtual environment, for example:</span></p><a id="t.c120c2f2e19056d8cd411c0cfc86657d0b83a7f8"></a><a id="t.14"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">cd </span><span class="c10">C</span><span class="c6">:</span><span class="c0">\Users\User\Scraper</span></p></td></tr></table><p class="c13"><span class="c12">Next, install the experimental Python Windows extensions module, </span><span class="c24 c12"><a class="c22" href="https://www.google.com/url?q=https://pypi.org/project/pypiwin32/&amp;sa=D&amp;source=editors&amp;ust=1680250635993231&amp;usg=AOvVaw36Bh71N6LYIG0mZz4qcBQv">pypiwin32</a></span><span class="c2 c11">. You&rsquo;ll also need to run the post-install script:</span></p><a id="t.bb2be6ce8b95c6c05534fbec1eed32eb4326b9e2"></a><a id="t.15"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c1">.</span><span class="c2 c0">\venv\Scripts\pip install pypiwin32</span></p><p class="c5"><span class="c1">.</span><span class="c0">\venv\Scripts\pywin32_postinstall</span><span class="c1">.</span><span class="c0">py </span><span class="c6">-</span><span class="c0">install</span></p></td></tr></table><p class="c13"><span class="c2 c11">Unfortunately, if you attempt to install your Python script as a Windows service with the current setup, you&rsquo;ll get the following error:</span></p><a id="t.8e8939ee240fb57d88e3550a7b11fedf99d070e5"></a><a id="t.16"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c6">****</span><span class="c0">&nbsp;</span><span class="c10">WARNING</span><span class="c0">&nbsp;</span><span class="c6">****</span></p><p class="c5"><span class="c0">The executable at </span><span class="c3">&quot;C:\Users\User\Scraper\venv\lib\site-packages\win32\PythonService.exe&quot;</span><span class="c0">&nbsp;is being used </span><span class="c4">as</span><span class="c0">&nbsp;a service</span><span class="c1">.</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">This executable doesn&#39;t have pythonXX</span><span class="c1">.</span><span class="c0">dll and</span><span class="c6">/</span><span class="c0">or pywintypesXX</span><span class="c1">.</span><span class="c0">dll </span><span class="c4">in</span><span class="c2 c0">&nbsp;the same</span></p><p class="c5"><span class="c0">directory</span><span class="c1">,</span><span class="c0">&nbsp;and they can&#39;t be found </span><span class="c4">in</span><span class="c0">&nbsp;the System directory</span><span class="c1">.</span><span class="c2 c0">&nbsp;This is likely to</span></p><p class="c5"><span class="c0">fail when used </span><span class="c4">in</span><span class="c0">&nbsp;the context </span><span class="c4">of</span><span class="c0">&nbsp;a service</span><span class="c1">.</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c2 c0">The exact environment needed will depend on which user runs the service and</span></p><p class="c5"><span class="c0">where Python is installed</span><span class="c1">.</span><span class="c0">&nbsp;If the service fails to run</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c4">this</span><span class="c0">&nbsp;will be why</span><span class="c1">.</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c10">NOTE</span><span class="c6">:</span><span class="c0">&nbsp;You should consider copying </span><span class="c4">this</span><span class="c2 c0">&nbsp;executable to the directory where these</span></p><p class="c5"><span class="c0">DLLs live </span><span class="c6">-</span><span class="c0">&nbsp;</span><span class="c3">&quot;C:\Users\User\Scraper\venv\lib\site-packages\win32&quot;</span><span class="c0">&nbsp;might be a good place</span><span class="c1">.</span></p></td></tr></table><p class="c13"><span class="c2 c11">However, if you follow the instructions of the error output, you&rsquo;ll be met with a new issue when trying to launch your script:</span></p><a id="t.2b8b6218712081dad85631a34eb9b96641ef5043"></a><a id="t.17"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">Error starting service</span><span class="c6">:</span><span class="c0">&nbsp;The service did not respond to the start or control request </span><span class="c4">in</span><span class="c0">&nbsp;a timely fashion</span></p></td></tr></table><p class="c13"><span class="c12">To solve this issue, you can add the Python libraries and interpreter to the Windows path. Alternatively, bundle your script and all its dependencies into an executable by using </span><span class="c14">pyinstaller</span><span class="c2 c11">:</span></p><a id="t.2834cc7866cdb07e16a04ac1cc0096a6ed17255a"></a><a id="t.18"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">venv\Scripts\pyinstaller </span><span class="c6">--</span><span class="c0">hiddenimport win32timezone </span><span class="c6">-</span><span class="c10">F</span><span class="c0">&nbsp;scrape</span><span class="c1">.</span><span class="c0">py</span></p></td></tr></table><p class="c17"><span class="c12">The </span><span class="c14">--</span><span class="c14">hiddenimport</span><span class="c14">&nbsp;</span><span class="c14">win32timezone</span><span class="c12">&nbsp;option is critical as the </span><span class="c14">win32timezone</span><span class="c2 c11">&nbsp;module is not explicitly imported but is still needed for the script to run.</span></p><p class="c13"><span class="c2 c11">Finally, let&rsquo;s install the script as a service and run it by invoking the executable you&rsquo;ve built previously:</span></p><a id="t.2ac7df0c68561929c129738fa3ed04dbe1125683"></a><a id="t.19"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c10">PS</span><span class="c0">&nbsp;</span><span class="c10">C</span><span class="c6">:</span><span class="c0">\Users\User\Scraper</span><span class="c6">&gt;</span><span class="c0">&nbsp;</span><span class="c1">.</span><span class="c0">\dist\scrape</span><span class="c1">.</span><span class="c2 c0">exe install</span></p><p class="c5"><span class="c2 c0">Installing service BookScraper</span></p><p class="c5"><span class="c2 c0">Changing service configuration</span></p><p class="c5"><span class="c2 c0">Service updated</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c10">PS</span><span class="c0">&nbsp;</span><span class="c10">C</span><span class="c6">:</span><span class="c0">\Users\User\Scraper</span><span class="c6">&gt;</span><span class="c0">&nbsp;</span><span class="c1">.</span><span class="c0">\dist\scrape</span><span class="c1">.</span><span class="c2 c0">exe start</span></p><p class="c5"><span class="c2 c0">Starting service BookScraper</span></p><p class="c5"><span class="c10">PS</span><span class="c0">&nbsp;</span><span class="c10">C</span><span class="c6">:</span><span class="c0">\Users\User\Scraper</span><span class="c6">&gt;</span></p></td></tr></table><p class="c17"><span class="c2 c11">And that&rsquo;s it. Now, you can open the Windows services utility and see your new service running.</span></p><p class="c17"><span class="c31">Protip:</span><span class="c12">&nbsp;you can read more about specific Windows API functions </span><span class="c12 c24"><a class="c22" href="https://www.google.com/url?q=https://docs.microsoft.com/en-us/windows/win32/sync/using-event-objects&amp;sa=D&amp;source=editors&amp;ust=1680250636002995&amp;usg=AOvVaw0juu9SRKp8Hiz2FyzdrTRk">here</a></span>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vwd37xqdk4m7lunfy8du.png)<span class="c2 c11">.</span></p><p class="c17"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 624.00px; height: 460.00px;"><img alt="" src="images/image7.png" style="width: 624.00px; height: 460.00px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span></p><h2 class="c28" id="h.w2hc6evxcsyi"><span class="c26 c11">Making your life easier by using NSSM on Windows</span></h2><p class="c25"><span class="c12">As evident, you can use </span><span class="c14">win32serviceutil</span><span class="c2 c11">&nbsp;to develop a Windows service. But the process is definitely not that simple &ndash; you could even say it sucks! Well, this is where the NSSM (Non-Sucking Service Manager) comes into play.</span></p><p class="c18"><span class="c2 c11">Let&rsquo;s simplify the script by only keeping the code that performs web scraping:</span></p><p class="c16 c18"><span class="c2 c11"></span></p><a id="t.b51bbc8e73c1119bcaa211d9c1ed93967c632352"></a><a id="t.20"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;json</span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;re</span></p><p class="c5"><span class="c0">from pathlib </span><span class="c4">import</span><span class="c2 c0">&nbsp;Path</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">import</span><span class="c2 c0">&nbsp;requests</span></p><p class="c5"><span class="c0">from bs4 </span><span class="c4">import</span><span class="c2 c0">&nbsp;BeautifulSoup</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">urls </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">[</span><span class="c3">&#39;https://books.toscrape.com/catalogue/sapiens-a-brief-history-of-humankind_996/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c3">&#39;https://books.toscrape.com/catalogue/shakespeares-sonnets_989/index.html&#39;</span><span class="c1">,</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; </span><span class="c3">&#39;https://books.toscrape.com/catalogue/sharp-objects_997/index.html&#39;</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c1">]</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">index </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c10">0</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c4">while</span><span class="c0">&nbsp;</span><span class="c10">True</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; url </span><span class="c6">=</span><span class="c0">&nbsp;urls</span><span class="c1">[</span><span class="c0">index </span><span class="c6">%</span><span class="c0">&nbsp;</span><span class="c8">len</span><span class="c1">(</span><span class="c0">urls</span><span class="c1">)]</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; index </span><span class="c6">+=</span><span class="c0">&nbsp;</span><span class="c10">1</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c8">print</span><span class="c1">(</span><span class="c3">&#39;Scraping url&#39;</span><span class="c1">,</span><span class="c0">&nbsp;url</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; response </span><span class="c6">=</span><span class="c0">&nbsp;requests</span><span class="c1">.</span><span class="c8">get</span><span class="c1">(</span><span class="c0">url</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; soup </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">BeautifulSoup</span><span class="c1">(</span><span class="c0">response</span><span class="c1">.</span><span class="c0">content</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;html.parser&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; book_name </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select_one</span><span class="c1">(</span><span class="c3">&#39;.product_main&#39;</span><span class="c1">).</span><span class="c0">h1</span><span class="c1">.</span><span class="c2 c0">text</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; rows </span><span class="c6">=</span><span class="c0">&nbsp;soup</span><span class="c1">.</span><span class="c8">select</span><span class="c1">(</span><span class="c3">&#39;.table.table-striped tr&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; product_info </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c1">{</span><span class="c0">row</span><span class="c1">.</span><span class="c0">th</span><span class="c1">.</span><span class="c0">text</span><span class="c6">:</span><span class="c0">&nbsp;row</span><span class="c1">.</span><span class="c0">td</span><span class="c1">.</span><span class="c0">text </span><span class="c4">for</span><span class="c0">&nbsp;row </span><span class="c4">in</span><span class="c0">&nbsp;rows</span><span class="c1">}</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; data_folder </span><span class="c6">=</span><span class="c0">&nbsp;</span><span class="c8">Path</span><span class="c1">(</span><span class="c3">&#39;C:\\Users\\User\\Scraper\\data&#39;</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; data_folder</span><span class="c1">.</span><span class="c8">mkdir</span><span class="c1">(</span><span class="c0">parents</span><span class="c6">=</span><span class="c0">True</span><span class="c1">,</span><span class="c0">&nbsp;exist_ok</span><span class="c6">=</span><span class="c0">True</span><span class="c1">)</span></p><p class="c5 c16"><span class="c2 c0"></span></p><p class="c5"><span class="c0">&nbsp; &nbsp; json_file_name </span><span class="c6">=</span><span class="c0">&nbsp;re</span><span class="c1">.</span><span class="c8">sub</span><span class="c1">(</span><span class="c3">&#39;[\&#39;: ]&#39;</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;-&#39;</span><span class="c1">,</span><span class="c0">&nbsp;book_name</span><span class="c1">)</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; json_file_path </span><span class="c6">=</span><span class="c0">&nbsp;data_folder </span><span class="c6">/</span><span class="c0">&nbsp;f</span><span class="c3">&#39;{json_file_name}.json&#39;</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; </span><span class="c4">with</span><span class="c0">&nbsp;</span><span class="c8">open</span><span class="c1">(</span><span class="c0">json_file_path</span><span class="c1">,</span><span class="c0">&nbsp;</span><span class="c3">&#39;w&#39;</span><span class="c1">)</span><span class="c0">&nbsp;</span><span class="c4">as</span><span class="c0">&nbsp;</span><span class="c10">book_file</span><span class="c6">:</span></p><p class="c5"><span class="c0">&nbsp; &nbsp; &nbsp; &nbsp; json</span><span class="c1">.</span><span class="c8">dump</span><span class="c1">(</span><span class="c0">product_info</span><span class="c1">,</span><span class="c0">&nbsp;book_file</span><span class="c1">)</span></p></td></tr></table><p class="c13"><span class="c12">Next, build a binary using </span><span class="c14">pyinstaller</span><span class="c2 c11">:</span></p><a id="t.2afc8e8cba77cdbd55d3c1b5ecbc4d43ca31cd8c"></a><a id="t.21"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">venv\Scripts\pyinstaller </span><span class="c6">-</span><span class="c10">F</span><span class="c0">&nbsp;simple_scrape</span><span class="c1">.</span><span class="c0">py</span></p></td></tr></table><p class="c17"><span class="c12">Now that you have a binary, it&rsquo;s time to install NSSM by visiting the </span><span class="c24 c12"><a class="c22" href="https://www.google.com/url?q=https://nssm.cc/download&amp;sa=D&amp;source=editors&amp;ust=1680250636014602&amp;usg=AOvVaw04F08nStjB3UNCXDGeuREe">official website</a></span><span class="c2 c11">. Extract it to a folder of your choice and add the folder to your PATH environment variable for convenience.</span></p>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3d5w87owuidzmf2j0ksp.png)<p class="c17"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 624.00px; height: 274.67px;"><img alt="" src="images/image2.png" style="width: 624.00px; height: 274.67px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span></p><p class="c17"><span class="c2 c11">Then, run the terminal as an admin.</span></p>
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fol5wtpo2dbz54ax5sa7.png)<p class="c17"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 624.00px; height: 586.67px;"><img alt="" src="images/image3.png" style="width: 624.00px; height: 586.67px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span></p><p class="c13"><span class="c2 c11">Once the terminal is open, change the directory to your script location:</span></p><a id="t.c120c2f2e19056d8cd411c0cfc86657d0b83a7f8"></a><a id="t.22"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">cd </span><span class="c10">C</span><span class="c6">:</span><span class="c0">\Users\User\Scraper</span></p></td></tr></table><p class="c13"><span class="c2 c11">Finally, install the script using NSSM and start the service:</span></p><a id="t.98379ef4273c1a8d00a263efe41246e508cef14f"></a><a id="t.23"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">nssm</span><span class="c1">.</span><span class="c0">exe install SimpleScrape </span><span class="c10">C</span><span class="c6">:</span><span class="c0">\Users\User\Scraper\dist\simple_scrape</span><span class="c1">.</span><span class="c2 c0">exe</span></p><p class="c5"><span class="c0">nssm</span><span class="c1">.</span><span class="c0">exe start SimpleScrape</span></p></td></tr></table><p class="c13"><span class="c31">Protip:</span><span class="c2 c11">&nbsp;if you have issues, redirect the standard error output of your service to a file to see what went wrong:</span></p><p class="c17 c16"><span class="c2 c11"></span></p><a id="t.ed7e0e3f8573e05fc3d0e5041ddd9de77454375d"></a><a id="t.24"></a><table class="c21"><tr class="c7"><td class="c20" colspan="1" rowspan="1"><p class="c5"><span class="c0">nssm</span><span class="c0">&nbsp;</span><span class="c4">set</span><span class="c0">&nbsp;SimpleScrape AppStderr </span><span class="c10">C</span><span class="c6">:</span><span class="c0">\Users\User\Scraper\service</span><span class="c6">-</span><span class="c0">error</span><span class="c1">.</span><span class="c0">log</span></p></td></tr></table><p class="c17"><span class="c2 c11">NSSM ensures that a service is running in the background, and if it doesn&rsquo;t, you at least get to know why.</span></p><h2 class="c28" id="h.4d5xbnsb6es"><span class="c11 c26">Conclusion</span></h2><p class="c25"><span class="c12">Regardless of the operating system, you have various options for setting up Python scripts for recurring web scraping tasks. Whether you need the configurability of </span><span class="c14">systemd</span><span class="c2 c11">, the flexibility of Windows services, or the simplicity of NSSM, be sure to follow this tried &amp; true guide as you navigate their features.</span></p><p class="c19"><span class="c12 c15 c33">If you have any questions about this tutorial or any other web scraping topics, don&#39;t hesitate to comment on this post, we&#39;ll answer you right away. </span></p></body></html>