Repository: scrapy/scrapy

Issue:
startproject only allows creating the project on an existing folder if the path is specified with an invalid module name
```
(venv) [adrian@afonsox temporal]$ mkdir existing_folder
(venv) [adrian@afonsox temporal]$ scrapy startproject existing_folder
Error: Module 'existing_folder' already exists
(venv) [adrian@afonsox temporal]$ cd existing_folder && scrapy startproject existing_folder .
New Scrapy project 'existing_folder', using template directory '/home/adrian/temporal/venv/lib/python3.8/site-packages/scrapy/templates/project', created in:
    /home/adrian/temporal/existing_folder

You can start your first spider with:
    cd .
    scrapy genspider example example.com
```


Discussion:
On master branch, I am able to startproject on existing_folder successfully.
```

(venv) rajesh@Rajesh-PC:~$ mkdir existing_folder
(venv) rajesh@Rajesh-PC:~$ scrapy startproject existing_folder
New Scrapy project 'existing_folder', using template directory '/home/rajesh/scrapy/scrapy/templates/project', created in:
    /home/rajesh/existing_folder

You can start your first spider with:
    cd existing_folder
    scrapy genspider example example.com

```
Weird. I can reproduce it with `master` as well:

```
[adrian@afonsox scrapy]$ git fetch upstream && git checkout upstream/master -B master
…
[adrian@afonsox scrapy]$  cd /home/adrian/temporal
[adrian@afonsox temporal]$ . venv/bin/activate
(venv) [adrian@afonsox temporal]$ pip install -e /home/adrian/proxectos/scrapy/
…
(venv) [adrian@afonsox temporal]$ mkdir existing_folder
(venv) [adrian@afonsox temporal]$ scrapy startproject existing_folder
Error: Module 'existing_folder' already exists
```
Not sure if I am missing any steps ( as I am new to the project). In the master branch I did the following:
```
sujith@kdekidd0  ~/mystudy/scrapy  virtualenv --python python3 .env
sujith@kdekidd0  ~/mystudy/scrapy  source .env/bin/activate
(.env)  sujith@kdekidd0  ~/mystudy/scrapy  pip install -e ~/test/scrapy
...
(.env)  sujith@kdekidd0  ~/mystudy/scrapy  mkdir existing_project # no errors as this is a new directory
(.env)  sujith@kdekidd0  ~/mystudy/scrapy  ./.env/bin/scrapy startproject existing_project
New Scrapy project 'existing_project', using template directory '/home/sujith/test/scrapy/scrapy/templates/project', created in:
    /home/sujith/mystudy/scrapy/existing_project

You can start your first spider with:
    cd existing_project
    scrapy genspider example example.com
(.env)  sujith@kdekidd0  ~/mystudy/scrapy  cd existing_project
(.env)  ✘ sujith@kdekidd0  ~/mystudy/scrapy/existing_project  ../.env/bin/scrapy startproject existing_project
Error: Module 'existing_project' already exists
(.env)  ✘ sujith@kdekidd0  ~/mystudy/scrapy/existing_project 
```
Nothing pops off my head. For a newly created directory I am able to execute the startproject with the templates. But when I switch inside the existing directory and execute the same error mentioned in the issue is shown up. Kindly help me know if this is the problem or my findings are incorrect.

Resolving patch:
diff --git a/scrapy/commands/startproject.py b/scrapy/commands/startproject.py
--- a/scrapy/commands/startproject.py
+++ b/scrapy/commands/startproject.py
@@ -1,7 +1,7 @@
 import re
 import os
 import string
-from importlib import import_module
+from importlib.util import find_spec
 from os.path import join, exists, abspath
 from shutil import ignore_patterns, move, copy2, copystat
 from stat import S_IWUSR as OWNER_WRITE_PERMISSION
@@ -42,11 +42,8 @@ def short_desc(self):
 
     def _is_valid_name(self, project_name):
         def _module_exists(module_name):
-            try:
-                import_module(module_name)
-                return True
-            except ImportError:
-                return False
+            spec = find_spec(module_name)
+            return spec is not None and spec.loader is not None
 
         if not re.search(r'^[_a-zA-Z]\w*$', project_name):
             print('Error: Project names must begin with a letter and contain'


Test patch:
diff --git a/tests/test_commands.py b/tests/test_commands.py
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -66,9 +66,14 @@ def call(self, *new_args, **kwargs):
 
     def proc(self, *new_args, **popen_kwargs):
         args = (sys.executable, '-m', 'scrapy.cmdline') + new_args
-        p = subprocess.Popen(args, cwd=self.cwd, env=self.env,
-                             stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                             **popen_kwargs)
+        p = subprocess.Popen(
+            args,
+            cwd=popen_kwargs.pop('cwd', self.cwd),
+            env=self.env,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            **popen_kwargs,
+        )
 
         def kill_proc():
             p.kill()
@@ -87,7 +92,10 @@ def kill_proc():
 class StartprojectTest(ProjectTest):
 
     def test_startproject(self):
-        self.assertEqual(0, self.call('startproject', self.project_name))
+        p, out, err = self.proc('startproject', self.project_name)
+        print(out)
+        print(err, file=sys.stderr)
+        self.assertEqual(p.returncode, 0)
 
         assert exists(join(self.proj_path, 'scrapy.cfg'))
         assert exists(join(self.proj_path, 'testproject'))
@@ -122,6 +130,25 @@ def test_startproject_with_project_dir(self):
         self.assertEqual(2, self.call('startproject'))
         self.assertEqual(2, self.call('startproject', self.project_name, project_dir, 'another_params'))
 
+    def test_existing_project_dir(self):
+        project_dir = mkdtemp()
+        project_name = self.project_name + '_existing'
+        project_path = os.path.join(project_dir, project_name)
+        os.mkdir(project_path)
+
+        p, out, err = self.proc('startproject', project_name, cwd=project_dir)
+        print(out)
+        print(err, file=sys.stderr)
+        self.assertEqual(p.returncode, 0)
+
+        assert exists(join(abspath(project_path), 'scrapy.cfg'))
+        assert exists(join(abspath(project_path), project_name))
+        assert exists(join(join(abspath(project_path), project_name), '__init__.py'))
+        assert exists(join(join(abspath(project_path), project_name), 'items.py'))
+        assert exists(join(join(abspath(project_path), project_name), 'pipelines.py'))
+        assert exists(join(join(abspath(project_path), project_name), 'settings.py'))
+        assert exists(join(join(abspath(project_path), project_name), 'spiders', '__init__.py'))
+
 
 def get_permissions_dict(path, renamings=None, ignore=None):
     renamings = renamings or tuple()