Repository: phpDocumentor/phpDocumentor (language: php)

Issue:
Problem including source code
# Template issue/request

I am having an issue including source code in the generated documentation. It may well be that I do not understand the functionality but there appears to be some sort of issue.  
  
If I use the **default** template there are not any files generated under the artifact name defined in _templates\default\template.xml_ (ie files/{{path}}.txt). I believe that this is because the source file _templates/default/source/file.source.txt.twig_ does not exists (I'm not sure why that twig file is defined under the sub-directory _source_ but that is a different matter).  
  
If I use the **clean** template then files are generated under the artifact name but they are all empty. That may be because the _template.xml_ for clean refers to **transformation writer="twig"** whereas the writer is **sourcecode** for the default template.  

As an aside, because of the PHAR it is difficult to make changes to test these theories but that may well be as a result of my ignorance!
  

- Version used: phpdocumentor-3.3.1
- Install method: PHIVE
- PHP version: PHP 8.1.4 (cli) (built: Mar 16 2022 09:33:31) (ZTS Visual C++ 2019 x64)
- Operating system and version: Windows 10
- 


Discussion / hints:
Did you add the `--sourcecode` flag in your command? 
I have **`<include-source>true</include-source>`** in my phpdoc.dist.xml. Is that equivalent?
it should be, however there could be a bug in this area :-) so I would like to ask you to give it a try. 
Thanks for this. I used the --sourcecode flag on the command line and things are starting to come together. With the default template some, but not all, of the source file were generated. All the source files were generated for the clean template. But in neither case did the files display - the error message ```✖ Error: File does not exist, is empty or trying to view from localhost``` is displayed instead. I'll investigate a bit and get back to you on what I think may be happening. Many thanks for your help.
I have a better idea of what is going on. As you have already explained you have to use the --sourcecode option.  
  
Using the default template, anything that isn't a Class doesn't display the source code. The source-view logic is missing from the generated file for all cases other than Classes (eg a file containing a single function doesn't display the source code):

```
        <div class="phpdocumentor-modal" id="source-view">
    <div class="phpdocumentor-modal-bg" data-exit-button></div>
    <div class="phpdocumentor-modal-container">
        <div class="phpdocumentor-modal-content">
        <pre style="max-height: 500px; overflow-y: scroll" data-src="files/src/core/classes/Debug.php.txt" class="language-php line-numbers linkable-line-numbers"></pre>
        </div>
        <button data-exit-button class="phpdocumentor-modal__close">X</button>
    </div>
</div>

    <script type="text/javascript">
        function loadExternalCodeSnippets(line) {
            Array.prototype.slice.call(document.querySelectorAll('pre[data-src]')).forEach((pre) => {
                var src = pre.getAttribute('data-src').replace( /\\/g, '/');
                var extension = (src.match(/\.(\w+)$/) || [, ''])[1];
                var language = 'php';

                var code = document.createElement('code');
                code.className = 'language-' + language;

                pre.textContent = '';

                pre.setAttribute('data-line', line)
                code.textContent = 'Loading…';

                pre.appendChild(code);

                var xhr = new XMLHttpRequest();

                xhr.open('GET', src, true);

                xhr.onreadystatechange = function () {
                    if (xhr.readyState == 4) {

                        if (xhr.status < 400 && xhr.responseText) {
                            code.textContent = xhr.responseText;

                            Prism.highlightElement(code);
                        }
                        else if (xhr.status >= 400) {
                            code.textContent = '✖ Error ' + xhr.status + ' while fetching file: ' + xhr.statusText;
                        }
                        else {
                            code.textContent = '✖ Error: File does not exist, is empty or trying to view from localhost';
                        }
                    }
                };

                xhr.send(null);
            });
        }

        var modals = document.querySelectorAll("[data-modal]");

        modals.forEach(function (trigger) {
            trigger.addEventListener("click", function (event) {
                //event.preventDefault();
                const modal = document.getElementById(trigger.dataset.modal);
                modal.classList.add("phpdocumentor-modal__open");
                loadExternalCodeSnippets(trigger.dataset.line)
                const exits = modal.querySelectorAll("[data-exit-button]");
                exits.forEach(function (exit) {
                    exit.addEventListener("click", function (event) {
                        event.preventDefault();
                        modal.classList.remove("phpdocumentor-modal__open");
                    });
                });
            });
        });
    </script>
```

Using the clean template, once you have drilled down into a file (ie select the FILE option in the right hand menu) then the code is displayed correctly but it doesn't display when trying to view the source from the list of functions under the top-level ToC.  

ENUMs are not documented using the clean template.

There isn't an issue to do with not all the files being generated, at least not one that I can reproduce, I'm afraid that you have to put that down to senility - I apologise for that.

The issues that you described for the default template should be resolved through PR https://github.com/phpDocumentor/phpDocumentor/pull/3284.

As for the reported issue on the clean template; I haven't checked that out. But I do admit that issues in the default template have more priority for me as I only have so much time :)

Fix patch:
diff --git a/data/templates/default/components/element-found-in.html.twig b/data/templates/default/components/element-found-in.html.twig
index bc09118b9c..8a45dd1466 100644
--- a/data/templates/default/components/element-found-in.html.twig
+++ b/data/templates/default/components/element-found-in.html.twig
@@ -4,6 +4,6 @@
     <span class="phpdocumentor-element-found-in__line">{{ node.line }}</span>
 
     {% if node.file.source %}
-        <a href="{{ target_path }}#source-view.{{ node.line }}" class="phpdocumentor-element-found-in__source" data-line="{{ node.line }}" data-modal="source-view" data-src="{{ path('files/' ~ node.path ~ '.txt')|raw }}"></a>
+        <a href="{{ target_path }}#source-view.{{ node.line }}" class="phpdocumentor-element-found-in__source" data-line="{{ node.line }}" data-modal="source-view" data-src="{{ path('files/' ~ node.file.path ~ '.txt')|raw }}"></a>
     {% endif %}
 </aside>
diff --git a/docs/references/full-config-reference.rst b/docs/references/full-config-reference.rst
index 1ac432ad74..a1c4868548 100644
--- a/docs/references/full-config-reference.rst
+++ b/docs/references/full-config-reference.rst
@@ -78,7 +78,7 @@ folder
 **Default**
 
 .. code-block:: xml
-  
+
 apis
 ~~~~
 
@@ -103,7 +103,7 @@ visibilities
 Visibilities are setting the deepest level of elements that will be rendered in your documentation. The values can be
 combined to fine tune the output. By default phpDocumentor includes all elements unless they are marked as ``@internal``
 
-Api will render only the elements marked as part of your api using the ``@api`` tag. 
+Api will render only the elements marked as part of your api using the ``@api`` tag.
 Public will filter all public elements from your project and render them in to the documentation.
 Protected will filter all public elements from your project and render them in to the documentation.
 Private will filter all public elements from your project and render them in to the documentation.
@@ -141,20 +141,20 @@ source
 ``````
 
 Where should phpDocumentor start looking for your code? In this configration section you can configure the DSN to your
-code base. 
+code base.
 
 The source can be configured using 2 parts. DSN which points to the root of your code base and paths which specify which
 folders should be read. Paths can be used to be more specific which folders should be included. All other paths in the root
-will be ignored. 
+will be ignored.
 
 .. note::
    Future versions of phpDocumentor will support other DSN formats rather then just local paths. Right now only
    `file://` is supported
 
-Paths do support glob patterns to be able to include only particular sub directories. 
+Paths do support glob patterns to be able to include only particular sub directories.
 The example below will only include files in your DSN root directory `src` and it's sub directories, when the file name matches
 `*Interface.php`
- 
+
 .. code-block:: xml
    <path>/src/**/*Interface.php</path>
 
@@ -213,17 +213,22 @@ include-source
 
 Should phpDocumentor include your source code in the rendered documentation? By including your source code people reading
 your docs can jump directly from the docs to a rendered version of the source code. Which allows them to get more details about
-a method or function implementation. 
+a method or function implementation.
+
+By default source code is not included in the rendered documentation.
+Unless the file contains a :doc:`@filesource <./phpdoc/tags/filesource>` tag.
+
+This setting can be overridden by using the ``--sourcecode`` or ``--no-sourcecode`` command line option.
 
 **Default**
 
 .. code-block:: xml
-   true
+   null
 
 examples
 ````````
 
-Examples are code snippets that can be included in your docblocks. This setting will configure where phpDocumentor 
+Examples are code snippets that can be included in your docblocks. This setting will configure where phpDocumentor
 can find them. The paths used in your docblock ``@example`` tags are relative to this dsn.
 For more information about ``@example`` please consult the tag reference.
 
@@ -264,20 +269,20 @@ source
 ``````
 
 Where should phpDocumentor start looking for your code? In this configuration section you can configure the DSN to your
-code base. 
+code base.
 
 The source can be configured using 2 parts. DSN which points to the root of your code base and paths which specify which
 folders should be read. Paths can be used to be more specific which folders should be included. All other paths in the root
-will be ignored. 
+will be ignored.
 
 .. note::
    Future versions of phpDocumentor will support other DSN formats rather then just local paths. Right now only
    `file://` is supported
 
-Paths do support glob patterns to be able to include only particular sub directories. 
+Paths do support glob patterns to be able to include only particular sub directories.
 The example below will only include files in your DSN root directory ``src`` and it's sub directories, when the file
 name matches ``*Interface.php``
- 
+
 .. code-block:: xml
    <path>/src/**/*Interface.php</path>
 
@@ -306,8 +311,8 @@ output
 settings
 --------
 
-Some settings are not part of the normal configuration parts of phpDocumentor. Because there have impact 
-on the full application behavior, or enable experimental parts in phpDocumentor. 
+Some settings are not part of the normal configuration parts of phpDocumentor. Because there have impact
+on the full application behavior, or enable experimental parts in phpDocumentor.
 
 Settings are key-value configuration options, which are listed with the following command:
 
diff --git a/src/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecode.php b/src/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecode.php
index e832ba0f82..256850b2b5 100644
--- a/src/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecode.php
+++ b/src/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecode.php
@@ -32,6 +32,10 @@ protected function process(ApiSetDescriptor $subject): ApiSetDescriptor
         }
 
         foreach ($subject->getFiles() as $file) {
+            if ($subject->getSettings()['include-source'] !== false && $file->getTags()->fetch('filesource') !== null) {
+                continue;
+            }
+
             $file->setSource(null);
         }
 
diff --git a/src/phpDocumentor/Configuration/ApiSpecification.php b/src/phpDocumentor/Configuration/ApiSpecification.php
index 10bf4fa4d0..deef9f6860 100644
--- a/src/phpDocumentor/Configuration/ApiSpecification.php
+++ b/src/phpDocumentor/Configuration/ApiSpecification.php
@@ -54,7 +54,7 @@ final class ApiSpecification implements ArrayAccess
     /** @var string */
     private $defaultPackageName;
 
-    /** @var bool */
+    /** @var bool|null */
     private $includeSource;
 
     /** @var array<string> */
@@ -86,7 +86,7 @@ private function __construct(
         array $extensions,
         array $visibility,
         string $defaultPackageName,
-        bool $includeSource,
+        bool|null $includeSource,
         array $markers,
         array $ignoreTags,
         ?Source $examples,
@@ -155,7 +155,7 @@ public static function createDefault(): ApiSpecification
             ['php'],
             [],
             '',
-            false,
+            null,
             [],
             [],
             null,
diff --git a/src/phpDocumentor/Configuration/Definition/Version3.php b/src/phpDocumentor/Configuration/Definition/Version3.php
index 0ec14f4053..edc5254bf9 100644
--- a/src/phpDocumentor/Configuration/Definition/Version3.php
+++ b/src/phpDocumentor/Configuration/Definition/Version3.php
@@ -302,7 +302,7 @@ private function apiSection(): ArrayNodeDefinition
                         ->end()
                     ->end()
                     ->booleanNode('include-source')
-                        ->defaultTrue()
+                        ->defaultNull()
                     ->end()
                     ->arrayNode('examples')
                         ->fixXmlConfig('path')
diff --git a/src/phpDocumentor/Console/Command/Project/RunCommand.php b/src/phpDocumentor/Console/Command/Project/RunCommand.php
index 6266a09057..ba10e8e44e 100644
--- a/src/phpDocumentor/Console/Command/Project/RunCommand.php
+++ b/src/phpDocumentor/Console/Command/Project/RunCommand.php
@@ -231,7 +231,7 @@ protected function configure(): void
             ->addOption(
                 'sourcecode',
                 null,
-                InputOption::VALUE_NONE,
+                InputOption::VALUE_NONE | InputOption::VALUE_NEGATABLE,
                 'Whether to include syntax highlighted source code'
             )
             ->addOption(
diff --git a/src/phpDocumentor/Transformer/Writer/Sourcecode.php b/src/phpDocumentor/Transformer/Writer/Sourcecode.php
index 7fff866af4..ef5172a277 100644
--- a/src/phpDocumentor/Transformer/Writer/Sourcecode.php
+++ b/src/phpDocumentor/Transformer/Writer/Sourcecode.php
@@ -50,10 +50,7 @@ public function transform(
         ProjectDescriptor $project,
         DocumentationSetDescriptor $documentationSet
     ): void {
-        if (
-            $documentationSet instanceof ApiSetDescriptor === false ||
-            $documentationSet->getSettings()['include-source'] === false
-        ) {
+        if ($documentationSet instanceof ApiSetDescriptor === false) {
             return;
         }
 


Test patch:
diff --git a/tests/unit/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecodeTest.php b/tests/unit/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecodeTest.php
index 7b793ec731..bdb34ce309 100644
--- a/tests/unit/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecodeTest.php
+++ b/tests/unit/phpDocumentor/Compiler/ApiDocumentation/Pass/RemoveSourcecodeTest.php
@@ -36,6 +36,21 @@ public function testRemovesSourceWhenDisabled(): void
     {
         $apiSetDescriptor = $this->faker()->apiSetDescriptor();
         $apiSetDescriptor = $this->givenFiles($apiSetDescriptor);
+        $apiSetDescriptor->getSettings()['include-source'] = false;
+        $fixture = new RemoveSourcecode();
+
+        $fixture->__invoke($apiSetDescriptor);
+
+        foreach ($apiSetDescriptor->getFiles() as $file) {
+            self::assertNull($file->getSource());
+        }
+    }
+
+    public function testRemovesSourceWhenNotSet(): void
+    {
+        $apiSetDescriptor = $this->faker()->apiSetDescriptor();
+        $apiSetDescriptor = $this->givenFiles($apiSetDescriptor);
+        $apiSetDescriptor->getSettings()['include-source'] = null;
         $fixture = new RemoveSourcecode();
 
         $fixture->__invoke($apiSetDescriptor);
@@ -62,6 +77,41 @@ public function testRemovesSourceWhenSourceShouldBeIncluded(): void
         }
     }
 
+    public function testSourceIsIncludedWhenFilesourceTagIsPresent(): void
+    {
+        $apiSetDescriptor = $this->faker()->apiSetDescriptor();
+        $apiSetDescriptor = $this->givenFiles($apiSetDescriptor);
+        $apiSetDescriptor->getFiles()->first()->getTags()->set(
+            'filesource',
+            $this->faker()->fileDescriptor()
+        );
+        $fixture = new RemoveSourcecode();
+
+        $fixture->__invoke($apiSetDescriptor);
+
+        foreach ($apiSetDescriptor->getFiles() as $file) {
+            self::assertNotNull($file->getSource());
+        }
+    }
+
+    public function testSourceIsRemovedWhenSettingDisabledExplicitly(): void
+    {
+        $apiSetDescriptor = $this->faker()->apiSetDescriptor();
+        $apiSetDescriptor = $this->givenFiles($apiSetDescriptor);
+        $apiSetDescriptor->getSettings()['include-source'] = false;
+        $apiSetDescriptor->getFiles()->first()->getTags()->set(
+            'filesource',
+            $this->faker()->fileDescriptor()
+        );
+        $fixture = new RemoveSourcecode();
+
+        $fixture->__invoke($apiSetDescriptor);
+
+        foreach ($apiSetDescriptor->getFiles() as $file) {
+            self::assertNull($file->getSource());
+        }
+    }
+
     private function givenFiles(ApiSetDescriptor $apiDescriptor): ApiSetDescriptor
     {
         $apiDescriptor->setFiles(
diff --git a/tests/unit/phpDocumentor/Configuration/Definition/Version3Test.php b/tests/unit/phpDocumentor/Configuration/Definition/Version3Test.php
index 53bd760055..68698c5d21 100644
--- a/tests/unit/phpDocumentor/Configuration/Definition/Version3Test.php
+++ b/tests/unit/phpDocumentor/Configuration/Definition/Version3Test.php
@@ -184,7 +184,7 @@ private function defaultConfigurationOutput(): array
                             'extensions' => [
                                 'extensions' => ['php', 'php3', 'phtml'],
                             ],
-                            'include-source' => true,
+                            'include-source' => false,
                             'validate' => false,
                             'markers' => ['markers' => ['TODO', 'FIXME']],
                             'ignore-tags' => ['ignore_tags' => []],
diff --git a/tests/unit/phpDocumentor/Transformer/Writer/SourcecodeTest.php b/tests/unit/phpDocumentor/Transformer/Writer/SourcecodeTest.php
index 64cd9ce3fd..d3b4843493 100644
--- a/tests/unit/phpDocumentor/Transformer/Writer/SourcecodeTest.php
+++ b/tests/unit/phpDocumentor/Transformer/Writer/SourcecodeTest.php
@@ -49,20 +49,6 @@ protected function setUp(): void
         );
     }
 
-    /**
-     * @covers ::transform
-     */
-    public function testNoInteractionWithTransformationWhenSourceShouldNotIncluded(): void
-    {
-        $transformation = $this->prophesize(Transformation::class);
-        $transformation->template()->shouldNotBeCalled();
-
-        $apiSet = $this->faker()->apiSetDescriptor();
-        $projectDescriptor = $this->giveProjectDescriptor($apiSet);
-
-        $this->sourceCode->transform($transformation->reveal(), $projectDescriptor, $apiSet);
-    }
-
     /**
      * @covers ::transform
      */
@@ -72,7 +58,6 @@ public function testNoInteractionWithTransformationWhenSourceIsIncluded(): void
         $transformation->template()->shouldBeCalled()->willReturn($this->faker()->template());
 
         $api = $this->faker()->apiSetDescriptor();
-        $api->getSettings()['include-source'] = true;
         $projectDescriptor = $this->giveProjectDescriptor($api);
 
         $this->sourceCode->transform($transformation->reveal(), $projectDescriptor, $api);