Repository: ManimCommunity/manim

Issue:
Broken source links on the stable version of the documentation
## Description of bug / unexpected behavior
Source links on the stable version of documentation does not work. It links to something like this: https://github.com/ManimCommunity/manim/blob/stable/manim/mobject/changing.py which is a 404 error. 

## Expected behavior
Source links should link to a file containing source code for the stable version. 

## How to reproduce the issue
On the documentation website, switch the version to stable. Navigate to and click the source link of any class. 

## Additional comments
Perhaps this is an access rights issue, which explains why it evaded detection from community devs for so long?

Render MarkupText colors using <span> tags
Currently, coloring uses `<color>` tags which are Manim-specific (unsupported by Pango), and were a workaround for lack of color parsing in the SVG engine. Now that SVG parsing for fill and color has improved (#915), MarkupText can pass a foreground color to Pango, and Pango's "official" coloring tag, `<span foreground="#rrggbb">`, can be used.

As discussed later in #915, there are two parts:
1. Passing in the fill color (which fill color?) to Pango as the `AttrForeground` attribute.
2. Removing the `init_colors` override for MarkupText objects.

cc @PhilippImhof 



Discussion:
The `stable` branch does not exist, that's why it showing the 404. @ManimCommunity/docs ?
Indeed, an oversight after getting rid of the `stable` branch. Can be fixed similarly to what is done here: https://github.com/ManimCommunity/manim/blob/6744c48c228c866c8f2256a7a2cd461bde6d411e/docs/source/conf.py#L118 -- possibly also just by linking to master, so adding `or ... == "stable"` to that line.

Sounds like an easy PR, does anyone here want to take care of it? :-)
I would like to do the PR (i am learning)
allow me?
> I would like to do the PR (i am learning)
> allow me?

Of course, please go ahead! :-)
I am not part of the project, is that why it does not allow me to do the PR? (sorry if I bother, I'm just learning and wanted to do my first PR)
![image](https://user-images.githubusercontent.com/68663197/108603885-2f86a880-7389-11eb-9964-62ec313fd5a4.png)

Please don't worry! Usually, the workflow with Github is a bit different: you create a pull request from your fork of the `ManimCommunity/manim` repository.

To create your fork, head to https://github.com/ManimCommunity/manim and look for the fork button on the top right of the page.
and from the fork I do the PR?
> and from the fork I do the PR?

Exactly. You can then choose to PR from your fork to the master branch of this repository.
can do it! also I was missing the title (it's my first time)
you are so friendly.  thank you very much for your help. I learned from this experience :)
I believe you accidentally made the PR to 3b1b/manim's master branch instead of ManimCommunity/manim's master branch:
![image](https://user-images.githubusercontent.com/29168469/108613270-20155880-73a5-11eb-975a-22a2b808c994.png)

thus there's a lot of merge conflicts (since 3b1b's version of manim is different). You should close the PR to 3b1b and make sure you're trying to merge your commits into ManimCommunity:master.
Is right. thanks for warning and sorry for the confusion
 I apologize, especially to @behackl, whoever gave me the opportunity to do the PR. but I can't do it correctly. I'm going to let someone with more experience than me do it. sorry for the loss of time and inconvenience. I must study more github
>  I apologize, especially to @behackl, whoever gave me the opportunity to do the PR. but I can't do it correctly. I'm going to let someone with more experience than me do it. sorry for the loss of time and inconvenience. I must study more github

Is there anything we can do to help you with getting started?
surely forget some step, because I can not do it.
I can see again what would be the steps to follow.
fix-1045 [(https://github.com/ManimCommunity/issues/1045)]
So now I have changed `MarkupText` in the following ways:

* pass `f'<span foreground="{self.color}">{self.text}</span>'` instead of just `self.text` to Pango in order to set the default color for all chars
* completely remove `def init_colors(...)` 

The SVG file is correctly created, e.g. all chars that are not in a particular `<span foreground="...">` are set in the default color (white in my case) and custom coloring of individual chars is correctly applied (gray background added manually just to make the white chars visible):
![image](https://user-images.githubusercontent.com/52650214/108620947-d291fd80-742f-11eb-91ab-5ca55da2147a.png)

However, the text does not appear in the animation:
![image](https://user-images.githubusercontent.com/52650214/108620959-e9d0eb00-742f-11eb-962f-a571ca22b041.png)



This is with the latest commits from PR #915. Is there still something on the SVG end that has to be changed?
What line exactly is the call to Pango? I tried 1287 but it gave me an attribute error - looks like self.color is unset at this point in the process. If I manually tell it to be white, then I do reproduce your bug.

I think this has to do with #1041, which requires a little bit of work on my end. I'm in the middle of it now.
Make the two changes (fill color + init_colors) to MarkupText on this branch - https://github.com/MrMallIronmaker/manim/tree/better-use

I was able to get the correct rendering that way, and included a test with an already-rendered version of the Pango MarkupText output. I haven't written a test for the MarkupText class yet, though.
> What line exactly is the call to Pango? I tried 1287 but it gave me an attribute error - looks like self.color is unset at this point in the process. If I manually tell it to be white, then I do reproduce your bug.

`self.color` used to be unset, because there was no need for it; I am setting it now in `MarkupText`'s `__init__`. Pango comes into play at the very end of the `text2svg` function. However, IMHO there is currently nothing to do over there, because the SVG is correctly created, e.g. with white (or whatever one chooses) as the default color.

> I think this has to do with #1041, which requires a little bit of work on my end. I'm in the middle of it now.

I'll wait for you to finish before pulling PR #915 again and (re-)doing my updates. I am just doing it in order to prepare the changes and have them ready once #915 is merged. OTOH, it makes no sense at all to actually create a PR with these changes before.
> Make the two changes (fill color + init_colors) to MarkupText on this branch - https://github.com/MrMallIronmaker/manim/tree/better-use

I'd rather just wait for you to finish your work on #915 and use this as the base as soon as it's merged. If I start a PR on your repo and we integrate that into your (already huge) PR, things just get more and more complicated IMHO. There's no need to hurry anyway, because all the functionality is already there; it will just become cleaner better for text with ligatures.
True true - only need to clone if you want to double-check the behavior is
correct - definitely no need for a PR.

On Mon, Feb 22, 2021 at 12:23 PM Philipp Imhof <notifications@github.com>
wrote:

> Make the two changes (fill color + init_colors) to MarkupText on this
> branch - https://github.com/MrMallIronmaker/manim/tree/better-use
>
> I'd rather just wait for you to finish your work on #915
> <https://github.com/ManimCommunity/manim/pull/915> and use this as the
> base as soon as it's merged. If I start a PR on your repo and we integrate
> that into your (already huge) PR, things just get more and more complicated
> IMHO. There's no need to hurry anyway, because all the functionality is
> already there; it will just become cleaner better for text with ligatures.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/ManimCommunity/manim/issues/1039#issuecomment-783650751>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAPG4TSUOTREGREVXZYHZGTTAK4NJANCNFSM4XW22DLQ>
> .
>

I have just retried using the most recent commits from #915. The SVG is again correctly created (e.g. white for uncolored chars), but the text does not show up. This is with `init_colors` removed from the `MarkupText` class.
It's likely the same issue as before - I didn't make any changes to #915 with regards to this bug. Sorry for the confusion. Do you think I should merge my better-use branch with #915?
Ah, then it was just a misunderstanding. I do not know how many changes that would make, you have already put in so much effort. On one hand, it would be good to have it fixed directly with the (hopefully soon-to-come) merge of #915, because I could then just file a PR to close this issue. (Instead of a joint PR or two separate PRs later on.) OTOH, it means more work for you and the reviewers, unless it's just a few lines of code.
It's a decent amout, unfortunately (https://github.com/MrMallIronmaker/manim/compare/master...MrMallIronmaker:better-use)

> showing 5 changed files with 115 additions and 39 deletions

Is there some way we can do a merge behind the scenes, and ultimately only make one PR to the community edition?
OMG, that's way too much to ask. It is probably the best if you add another PR later on. I should then be able to push commits to that PR inside this repo. That's certainly the easiest way to go, especially because (as I suppose) there are not many changes for the `MarkupText` part.
Will do. I can either make the PR now and rebase when #915 is merged, or wait until #915 is merged to make the PR. Do you have a preference?
I'd say your preference will be mine; you are the one doing the tough job :)
PR #1058 has been opened!

Resolving patch:
diff --git a/scripts/dev_changelog.py b/scripts/dev_changelog.py
new file mode 100755
--- /dev/null
+++ b/scripts/dev_changelog.py
@@ -0,0 +1,254 @@
+#!/usr/bin/env python
+"""Script to generate contributor and pull request lists
+This script generates contributor and pull request lists for release
+changelogs using Github v3 protocol. Use requires an authentication token in
+order to have sufficient bandwidth, you can get one following the directions at
+`<https://help.github.com/articles/creating-an-access-token-for-command-line-use/>_
+Don't add any scope, as the default is read access to public information. The
+token may be stored in an environment variable as you only get one chance to
+see it.
+
+Usage::
+    $ ./scripts/dev_changelog.py <token> <revision range> <output_file>
+
+The output is utf8 rst.
+
+Dependencies
+------------
+- gitpython
+- pygithub
+
+Examples
+--------
+From the bash command line with $GITHUB token::
+    $ ./scripts/dev_changelog.py $GITHUB v0.3.0..v0.4.0 -o 0.4.0-changelog.rst
+
+Note
+----
+This script was taken from Numpy under the terms of BSD-3-Clause license.
+"""
+import re
+import sys
+from collections import defaultdict
+from textwrap import dedent, indent
+from pathlib import Path
+import git
+from tqdm import tqdm
+from git import Repo
+from github import Github
+
+
+this_repo = Repo(str(Path(__file__).resolve().parent.parent))
+
+
+def get_authors_and_reviewers(revision_range, github_repo, pr_nums):
+    pat = r"^.*\t(.*)$"
+    lst_release, cur_release = [r.strip() for r in revision_range.split("..")]
+
+    # authors, in current release and previous to current release.
+    cur = set(re.findall(pat, this_repo.git.shortlog("-s", revision_range), re.M))
+    pre = set(re.findall(pat, this_repo.git.shortlog("-s", lst_release), re.M))
+
+    # Append '+' to new authors.
+    authors = [s + " +" for s in cur - pre] + [s for s in cur & pre]
+    authors.sort()
+
+    reviewers = []
+    for num in tqdm(pr_nums, desc="Fetching reviewer comments"):
+        pr = github_repo.get_pull(num)
+        reviewers.extend(rev.user.name for rev in pr.get_reviews())
+    reviewers = sorted(set(rev for rev in reviewers if rev is not None))
+
+    return {"authors": authors, "reviewers": reviewers}
+
+
+def get_pr_nums(revision_range):
+    print("Getting PR Numbers:")
+    prnums = []
+
+    # From regular merges
+    merges = this_repo.git.log("--oneline", "--merges", revision_range)
+    issues = re.findall(r".*\(\#(\d+)\)", merges)
+    prnums.extend(int(s) for s in issues)
+
+    # From fast forward squash-merges
+    commits = this_repo.git.log(
+        "--oneline", "--no-merges", "--first-parent", revision_range
+    )
+    issues = re.findall(r"^.*\(\#(\d+)\)$", commits, re.M)
+    prnums.extend(int(s) for s in issues)
+
+    print(prnums)
+    return prnums
+
+
+def sort_by_labels(github_repo, pr_nums):
+    """Sorts PR into groups based on labels.
+
+    This implementation sorts based on importance into a singular group. If a
+    PR uses multiple labels, it is sorted under one label.
+
+    The importance order (for the end-user):
+    - breaking changes
+    - highlight
+    - feature
+    - enhancement
+    - bug
+    - documentation
+    - testing
+    - infrastructure
+    - unlabeled
+    """
+    pr_by_labels = defaultdict(list)
+    for num in tqdm(pr_nums, desc="Sorting by labels"):
+        pr = github_repo.get_pull(num)
+        labels = [label.name for label in pr.labels]
+        # TODO: Make use of label names directly from main
+        if "breaking changes" in labels:
+            pr_by_labels["breaking changes"].append(pr)
+        elif "highlight" in labels:
+            pr_by_labels["highlight"].append(pr)
+        elif "new feature" in labels:
+            pr_by_labels["new feature"].append(pr)
+        elif "enhancement" in labels:
+            pr_by_labels["enhancement"].append(pr)
+        elif "bug" in labels:
+            pr_by_labels["bug"].append(pr)
+        elif "deprecation" in labels:
+            pr_by_labels["deprecation"].append(pr)
+        elif "documentation" in labels:
+            pr_by_labels["documentation"].append(pr)
+        elif "release" in labels:
+            pr_by_labels["release"].append(pr)
+        elif "testing" in labels:
+            pr_by_labels["testing"].append(pr)
+        elif "infrastructure" in labels:
+            pr_by_labels["infrastructure"].append(pr)
+        elif "maintenance":
+            pr_by_labels["maintenance"].append(pr)
+        elif "style" in labels:
+            pr_by_labels["style"].append(pr)
+        else:  # PR doesn't have label :( Create one!
+            pr_by_labels["unlabeled"].append(pr)
+
+    return pr_by_labels
+
+
+def get_summary(body):
+    pattern = '<!--changelog-start-->([^"]*)<!--changelog-end-->'
+    has_changelog_pattern = re.search(pattern, body)
+    if has_changelog_pattern:
+
+        return has_changelog_pattern.group()[22:-21].strip()
+
+
+def main(token, revision_range, outfile=None):
+    lst_release, cur_release = [r.strip() for r in revision_range.split("..")]
+
+    github = Github(token)
+    github_repo = github.get_repo("ManimCommunity/manim")
+
+    pr_nums = get_pr_nums(revision_range)
+
+    # document authors
+    contributors = get_authors_and_reviewers(revision_range, github_repo, pr_nums)
+    authors = contributors["authors"]
+    reviewers = contributors["reviewers"]
+
+    if not outfile:
+        outfile = (
+            Path(__file__).resolve().parent.parent / "docs" / "source" / "changelog"
+        )
+        outfile = outfile / f"{cur_release[1:]}-changelog.rst"
+    else:
+        outfile = Path(outfile).resolve()
+
+    with outfile.open("w", encoding="utf8") as f:
+        heading = "Contributors"
+        f.write(f"{heading}\n")
+        f.write("=" * len(heading) + "\n\n")
+        f.write(
+            dedent(
+                f"""\
+                A total of {len(set(authors).union(set(reviewers)))} people contributed to this
+                release. People with a '+' by their names authored a patch for the first
+                time.\n
+                """
+            )
+        )
+
+        for author in authors:
+            f.write(f"* {author}\n")
+
+        f.write("\n")
+        f.write(
+            dedent(
+                """
+                The patches included in this release have been reviewed by
+                the following contributors.\n
+                """
+            )
+        )
+
+        for reviewer in reviewers:
+            f.write(f"* {reviewer}\n")
+
+        # document pull requests
+        heading = "Pull requests merged"
+        f.write("\n")
+        f.write(heading + "\n")
+        f.write("=" * len(heading) + "\n\n")
+        f.write(
+            f"A total of {len(pr_nums)} pull requests were merged for this release.\n\n"
+        )
+
+        # TODO: Use labels list in sort_by_labels, simplify logic
+        labels = [
+            "breaking changes",
+            "highlight",
+            "new feature",
+            "enhancement",
+            "bug",
+            "deprecation",
+            "documentation",
+            "release",
+            "testing",
+            "infrastructure",
+            "maintenance",
+            "style",
+            "unlabeled",
+        ]
+        pr_by_labels = sort_by_labels(github_repo, pr_nums)
+        for label in labels:
+            pr_of_label = pr_by_labels[label]
+
+            if pr_of_label:
+                f.write(f"{label.capitalize()}\n")
+                f.write("-" * len(label) + "\n\n")
+
+                for PR in pr_by_labels[label]:
+                    num = PR.number
+                    url = PR.html_url
+                    title = PR.title
+                    label = PR.labels
+                    f.write(f"* `#{num} <{url}>`__: {title}\n")
+                    overview = get_summary(PR.body)
+                    if overview:
+                        f.write(indent(f"{overview}\n", "   "))
+                    else:
+                        f.write("\n")
+
+    print(f"Wrote changelog to: {outfile}")
+
+
+if __name__ == "__main__":
+    from argparse import ArgumentParser, FileType
+
+    parser = ArgumentParser(description="Generate author/pr lists for release")
+    parser.add_argument("token", help="github access token")
+    parser.add_argument("revision_range", help="<revision>..<revision>")
+    parser.add_argument(
+        "-o", "--outfile", type=str, help="path and file name of the changelog output"
+    )
+    args = parser.parse_args()
+    main(args.token, args.revision_range, args.outfile)


Test patch: