Localization of scikit-learn website content.

Hi,

I work for Quansight Labs and am helping set things up for translation of main project website content for core projects in the Scientific Python ecosystem. This work is supported by the [Scientific Python Community & Communications Infrastructure grant](https://scientific-python.org/doc/scientific-python-community-and-communications-infrastructure-2022.pdf) from CZI. I created a Github discussion for this here: https://github.com/scikit-learn/scikit-learn/discussions/28105, but @glemaitre pointed out that an issue would be more appropriate.

A deliverable for this grant is to have the brochure websites of at least 8 of the 10 Scientific Python core projects translated into at least 3 commonly used languages. You may have seen the language drop-down selector at https://numpy.org/.  The goal is for a cross functional team of Quansight staff and volunteers to handle the bulk of the work, taking the burden off of project maintainers.

The localization management platform [Crowdin](https://crowdin.com/) is offering a free supported enterprise organization for Scientific Python translations. This is the platform we used for numpy.org. At this moment you would not need to decide what if anything you would do with translated content. *However, to get things started, I'd like to ask permission to fill out the [Crowdin Open Source Project Setup Request ](https://crowdin.com/page/open-source-project-setup-request) form on your behalf, to allow me to add a project to the Scientific Python Crowdin organization for Scikit-Learn.* 

For those interested. Here are the specifics of what I'd do with this.

- Create a GitHub repository which mirrors the content from the Scikit-Learn brochure website.
- Setup a cron github action which polls for updates to the Scikit-Learn website content and helps keep the mirror up to date.
- Sync this repository to the Scikit-Learn Crowdin project. Translate can then translate the content, and Crowdin will automatically push commits to a PR against the repository mirroring the website content.

The end product will be a repository with parallel versions of the website content in a variety of languages which is kept up to date. If you want to host the translations on your webpage like NumPy does, myself and/or a colleague from Quansight would be supported by the grant to help make that happen.

Please let me know if you have any questions.

COMMENT:
> I'm not against creating an account, but I think it is worth having a bit more of a coherent plan instead of salami slicing the asks so that each one is easy to agree to.

I missed this. Sorry if I gave that impression. I don't plan to salami slice this. My hope is that there will be one small salami slice, so I can get everything set up, which will be easier to do if I can set every project up at once. After that it will just be discussion of the rest of the salami. I think milestones in the grant offer some flexibility for what is actually accomplished and the plan can be adapted to each projects needs.

I can try to give an overview of the plan as I understand it.

First the scope for content is limited to only information on the core project website which does not change too frequently. The scope for languages is limited to widely spoken languages with a relatively low percentage of speakers proficient in English. The idea is that the amount of content to translate for one project into one language should only be a few days worth of work total.

To summarize what's happened so far. As a proof of concept, we set up translation infrastructure for https://numpy.org (managed by PIs on the grant), ha content translated, and publish translations. This is done. Portuguese and Japanese translations are available now. Korean translations will be published within the next few weeks.

After this I started looking into how things could be set up for other projects.  When I started reaching out to other projects, Ididn't have a clear plan yet. Or even a good idea of what I was doing. I had a good discussion with the Pandas developers, from which I realized how important it really is to keep the work needed from core project maintainers to a minimum.

Now I'm trying to get the translation infrastructure set up for the different projects. I have a technical solution for this, that I described above. Basically, having repositories that mirror the content which are synced to Crowdin, which will allow for maintaining parallel versions of the content translated into different languages. The idea here is that there would be a GitHub action in the mirror repo which polls for changes in the content in the source repo and pulls them in when needed. Crowdin will then be aware of these updates, and will push notifications to the translators that there is fresh content that needs to be translated. When translations are made, Crowdin will push commits to a branch on the repo its synced with adding the updates to the translations. I have a workflow for submitting PRs to update to translations and we have admins for each language who can give a final sign off.

For numpy.org, Crowdin is synced directly with the associated Github repo, https://github.com/numpy/numpy.org, so when such a PR is merged, the translations get updated online automatically. If the translated content appears in a separate repo, extra steps will be needed to actual publish the content on the Scikit-Learn website. The Pandas developers maintain their own static site generator, and my undestanding is their plan would involve pulling down content from the mirror repo dynamically when the website is build, and putting parallel versions at places like `pandas.pydata.org/pt`. It seems every core project website is a bit different, so how the translations actually appear on the webpage would differ between projects.

For keeping things up to date in the long term, like I mentioned, the goal is to build a community of translators who can help this. The same group of translators could work on projects across the Scientific Python ecosystem, and having all of the projects in the same Crowdin organization means there would be no friction between someone trying to translate for one project and then moving to another. After the ground work is set down, in about one month, someone is going to take over with experience with this kind of work. I'm not really qualified to judge the feasibility of accomplishing this, but it seems reasonable to me based on my experience watching over how things have gone with the NumPy translations.

Labels: Needs Decision