Repository: getnikola/nikola

Issue:
Complete URL translation
Currently there seems to be no way to completely translate a post URL, while keeping the "Also available in:" relation among translations.

The source files are kept together and have to be named in the default language to be connected: "blog_post.en.rst" and "blog_post.it.rst", for instance. They also have to be placed in the same directory.

The section folder is not translated, even when setting the section in metadata.

The slug can be translated by setting it in metadata.

See also issue #1914.



Discussion:
Maybe we could expand TRANSLATIONS_PATTERN?

Right now it looks like this:

```
TRANSLATIONS_PATTERN = "{path}.{lang}.{ext}"
```

Maybe it could be `{section}/{path}.{lang}.{ext}` or something... 

There is also FILE_METADATA_REGEXP that may be abused...

I'm currently playing with `destination_path()` in `post.py` to translate URLs. My real case has several output folders and several languages, but for testing I'm only using one output folder in two languages.

I'm defining localized output folders like this: (for testing, this is directly in the `destination_path()` function, but it could be moved to `conf.py`) 

```
translated_destinations = {
    'articles': {
        'en': 'articles',
        'pt': 'artigos'
    }
}
```

And then, at the beginning of the `destination_path()` function, I try to get the localized folder name with:

```
try:
    localized_folder = translated_destinations[self.folder][lang]
except KeyError:
    localized_folder = self.folder
```

and I use `localized_folder` instead of `self.folder` when constructing the `path`.

This is somewhat working: the files are generated properly, and I have URLs like
`output/articles/slug-for-english/`
`output/artigos/slug-for-portuguese/`

However, on the English page, the interlanguage link ("Also available in: Portuguese") links to: 
`output/articles/slug-for-portuguese`
which also exists, so there's something else that needs to be changed.

I'd appreciate guidance from someone more familiar with the Nikola code to find what else needs to be changed, so that that last duplicated page isn't generated, and the interlanguage link points to the right place.

(Maybe there's a better way to do all this; if so, feel free to point me in the correct direction. I just want to find a way to translate output folders :) )

You can already set output directory by language by specifying different source folders or file patterns. For example:

`POSTS = (("posts/*.pt.md", "artigos", "post.tmpl"), ("posts/*.en.md", "articles", "post.tmpl"))`

See https://irclogs.getnikola.com/logs/2015-09-23/

@Aeyoun Thank you for responding.

The method you suggest was the first I tried; unfortunately, unless I'm doing something wrong, it doesn't work, because it seems that translation candidates are returned regardless of the wildcard.

If I have two posts `posts/title.en.md` and `posts/title.pt.md`, and I only add the English wildcard like this:
`POSTS = (("posts/*.en.md", "articles", "post.tmpl"))`
then the `en` post is generated, but the `pt` post is _also_ generated, even if it doesn't match the wildcard, because it's a translation of the `en` post.

So if I add one wildcard per language like you suggest:
`POSTS = (("posts/*.pt.md", "artigos", "post.tmpl"), ("posts/*.en.md", "articles", "post.tmpl"))`
I get duplication errors because because the translated posts are attempted to be created twice: once on their own, and once as translations of the `en` posts.

It's of course possible that I'm mistaken. If anyone has a working example of the approach you suggest (whether a minimum working example, or the code of a live site), I'll gladly try to reproduce it, and update the documentation accordingly.

**Edit:** It was [confirmed on the list](https://groups.google.com/d/msg/nikola-discuss/NFvu-8Htr5M/kLtAX10TAQAJ) that this isn't possible yet.

How about allowing to override the path with `.. path: translated/path/to/post` in the localized metadata? Then the files are in one place (like the path for the main language), but the compiled results will end up where you want them.

This would be similar to `.. slug:`, which allows you to overwrite the slug generated out of the post's title.


Resolving patch:
diff --git a/nikola/plugins/misc/scan_posts.py b/nikola/plugins/misc/scan_posts.py
--- a/nikola/plugins/misc/scan_posts.py
+++ b/nikola/plugins/misc/scan_posts.py
@@ -55,10 +55,10 @@ def scan(self):
                 self.site.config['post_pages']:
             if not self.site.quiet:
                 print(".", end='', file=sys.stderr)
+            destination_translatable = utils.TranslatableSetting('destination', destination, self.site.config['TRANSLATIONS'])
             dirname = os.path.dirname(wildcard)
             for dirpath, _, _ in os.walk(dirname, followlinks=True):
-                dest_dir = os.path.normpath(os.path.join(destination,
-                                            os.path.relpath(dirpath, dirname)))  # output/destination/foo/
+                rel_dest_dir = os.path.relpath(dirpath, dirname)
                 # Get all the untranslated paths
                 dir_glob = os.path.join(dirpath, os.path.basename(wildcard))  # posts/foo/*.rst
                 untranslated = glob.glob(dir_glob)
@@ -93,11 +93,12 @@ def scan(self):
                         post = Post(
                             base_path,
                             self.site.config,
-                            dest_dir,
+                            rel_dest_dir,
                             use_in_feeds,
                             self.site.MESSAGES,
                             template_name,
-                            self.site.get_compiler(base_path)
+                            self.site.get_compiler(base_path),
+                            destination_base=destination_translatable
                         )
                         timeline.append(post)
                     except Exception as err:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -87,13 +87,17 @@ def __init__(
         use_in_feeds,
         messages,
         template_name,
-        compiler
+        compiler,
+        destination_base=None
     ):
         """Initialize post.
 
         The source path is the user created post file. From it we calculate
         the meta file, as well as any translations available, and
         the .html fragment file path.
+
+        destination_base must be None or a TranslatableSetting instance. If
+        specified, it will be prepended to the destination path.
         """
         self.config = config
         self.compiler = compiler
@@ -120,9 +124,10 @@ def __init__(
         # cache/posts/blah.html
         self._base_path = self.base_path.replace('\\', '/')
         self.metadata_path = self.post_name + ".meta"  # posts/blah.meta
-        self.folder = destination
-        self.translations = self.config['TRANSLATIONS']
+        self.folder_relative = destination
+        self.folder_base = destination_base
         self.default_lang = self.config['DEFAULT_LANG']
+        self.translations = self.config['TRANSLATIONS']
         self.messages = messages
         self.skip_untranslated = not self.config['SHOW_UNTRANSLATED_POSTS']
         self._template_name = template_name
@@ -161,6 +166,18 @@ def __init__(
             for lang in sorted(self.translated_to):
                 default_metadata.update(self.meta[lang])
 
+        # Compose paths
+        if self.folder_base is not None:
+            # Use translatable destination folders
+            self.folders = {}
+            for lang in self.config['TRANSLATIONS'].keys():
+                self.folders[lang] = os.path.normpath(os.path.join(
+                    self.folder_base(lang), self.folder_relative))
+        else:
+            # Old behavior (non-translatable destination path, normalized by scanner)
+            self.folders = {lang: self.folder_relative for lang in self.config['TRANSLATIONS'].keys()}
+        self.folder = self.folders[self.default_lang]
+
         # Load data field from metadata
         self.data = Functionary(lambda: None, self.default_lang)
         for lang in self.translations:
@@ -768,12 +785,13 @@ def destination_path(self, lang=None, extension='.html', sep=os.sep):
         """
         if lang is None:
             lang = nikola.utils.LocaleBorg().current_lang
+        folder = self.folders[lang]
         if self._has_pretty_url(lang):
             path = os.path.join(self.translations[lang],
-                                self.folder, self.meta[lang]['slug'], 'index' + extension)
+                                folder, self.meta[lang]['slug'], 'index' + extension)
         else:
             path = os.path.join(self.translations[lang],
-                                self.folder, self.meta[lang]['slug'] + extension)
+                                folder, self.meta[lang]['slug'] + extension)
         if sep != os.sep:
             path = path.replace(os.sep, sep)
         if path.startswith('./'):
@@ -845,7 +863,7 @@ def permalink(self, lang=None, absolute=False, extension='.html', query=None):
             extension = self.compiler.extension()
 
         pieces = self.translations[lang].split(os.sep)
-        pieces += self.folder.split(os.sep)
+        pieces += self.folders[lang].split(os.sep)
         if self._has_pretty_url(lang):
             pieces += [self.meta[lang]['slug'], 'index' + extension]
         else:


Test patch: