Repository: conan-io/conan

Issue:
New unix2dos tool #2605
- [x] Refer to the issue that supports this Pull Request:
- This PR references the issue #2605.

- [x] If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
- No missing information.

- [x] I've read the [Contributing guide](https://raw.githubusercontent.com/conan-io/conan/develop/.github/CONTRIBUTING.md).
- [x] I've followed the PEP8 style guides for  #Python code.
- [x] I've opened another PR in the Conan docs repo to the ``develop`` branch, documenting this one. Also adding a description of the changes in the ``changelog.rst`` file. https://github.com/conan-io/docs
- PR [#596](https://github.com/conan-io/docs/pull/596)


Discussion:
[![CLA assistant check](https://cla-assistant.io/pull/badge/signed)](https://cla-assistant.io/conan-io/conan?pullRequest=2655) <br/>All committers have signed the CLA.
Hello,

I was not aware of the existence of the `normalize()` function, I could not find it in the documentation.
It seems to provide the same kind of service, but the `unix2dos()` function is intended to be used directly to convert a file instead of a text stream.

I tried to stick to the behavior of the well known [Unix2dos](https://en.wikipedia.org/wiki/Unix2dos) binary on Unix, I did not condition it on Windows because if you need call this tool you should know what you are doing (I cannot think of a use case of this on Linux, but who knows?).

In my situation, the usage of this tool is describe in the issue #2605, we are building packages on both Linux and Windows and we faced issues with line endings in our conan packages.. We worked around the issue by enforcing a git configuration of `autocrlf = false` on all our Windows servers but I had a corner case where I was creating a Visual Studio project on the fly by applying a git patch. This file was created with Unix line endings and could not be opened by the devenv as a result.
1. it probably make sense to provide dos2unix in addition of unix2dos
2. suggested code will convert '\n\r' into '\n\n\r', which is incorrect
I have introduced a new `dos2unix` function to mirror the proposed `unix2dos` one.

@SSE4, Indeed, but such a use case would only occur with someone using the `unix2dos` function on a DOS file instead of an Unix one, which is functionally incorrect. IMO, developers using these kind of functions are expected to know what they are doing. Nevertheless, I have tried to add some checks in the proposed functions to prevent using them with the wrong file input format.
It doesn't feel very robust.
Should be better to use `str.splitlines()` and `"\n".join(lines)` to take advantage of the [universal endlines](https://docs.python.org/2/glossary.html#term-universal-newlines)?
And probably to deal with the encoding, probably it is a good idea to use the current `conans.util.files` methods:

```
from conans.util.files import load, save

def unix2dos(filepath):
    save(filepath, "\r\n".join(load(filepath).splitlines()))

def dos2unix(filepath):
    save(filepath, "\n".join(load(filepath).splitlines()))
```

Sorry for the late reply, I must have missed the notification.
I understand that if feels a bit weak, but I am not sure how do it otherwise.
I have tried your solution but I think that passing `\r\n` and `\n` as strings instead of bytes "hides" them.
If I use my `new_unix_file` or `new_dos_file` functions (working with bytes), the resulting files are detected as  expected:

    $ file "/tmp/tmpy7i72cyfconans/path with spaces/unix_file1"
    /tmp/tmpy7i72cyfconans/path with spaces/unix_file1: ASCII text
    $ file "/tmp/tmpp0upjjnxconans/path with spaces/dos_file2"
    /tmp/tmpp0upjjnxconans/path with spaces/dos_file2: ASCII text, with CRLF line terminators

However, when working with strings like in your example, the files are considered as `ASCII text, with no 
line terminators`, making it impossible to test for conversion.
Running it through hexdump shows the missing characters:

```
/tmp/tmpt9ph8uhoconans/path with spaces/unix_file1: ASCII text
00000000  54 68 69 73 20 69 73 20  61 20 74 65 73 74 0a     |This is a test.|
0000000f
/tmp/tmpt9ph8uhoconans/path with spaces/unix_file1: ASCII text, with no line terminators
00000000  54 68 69 73 20 69 73 20  61 20 74 65 73 74        |This is a test|
0000000e
F/tmp/tmp8ssvyrpeconans/path with spaces/dos_file1: ASCII text, with CRLF line terminators
00000000  54 68 69 73 20 69 73 20  61 20 74 65 73 74 0d 0a  |This is a test..|
00000010
/tmp/tmp8ssvyrpeconans/path with spaces/dos_file1: ASCII text, with no line terminators
00000000  54 68 69 73 20 69 73 20  61 20 74 65 73 74        |This is a test|
0000000e
```
Ok, we will have a peep to your comment, I would like to try and understand why y proposal is not working before taking a decision. Thanks!

Resolving patch:
diff --git a/conans/client/tools/files.py b/conans/client/tools/files.py
--- a/conans/client/tools/files.py
+++ b/conans/client/tools/files.py
@@ -297,3 +297,19 @@ def _get_possible_filenames(filename):
                         return trick_path
 
     return None
+
+
+def _replace_with_separator(filepath, sep):
+    tmp = load(filepath)
+    ret = sep.join(tmp.splitlines())
+    if tmp.endswith("\n"):
+        ret += sep
+    save(filepath, ret)
+
+
+def unix2dos(filepath):
+    _replace_with_separator(filepath, "\r\n")
+
+
+def dos2unix(filepath):
+    _replace_with_separator(filepath, "\n")


Test patch:
diff --git a/conans/test/util/tools_test.py b/conans/test/util/tools_test.py
--- a/conans/test/util/tools_test.py
+++ b/conans/test/util/tools_test.py
@@ -1235,3 +1235,72 @@ def build(self):
         client.save({"conanfile.py": conanfile, "other": "hello"})
         client.run("create . user/channel", ignore_error=True)
         self.assertIn("specify a branch to checkout", client.out)
+
+    def unix_to_dos_unit_test(self):
+
+        def save_file(contents):
+            tmp = temp_folder()
+            filepath = os.path.join(tmp, "a_file.txt")
+            save(filepath, contents)
+            return filepath
+
+        fp = save_file(b"a line\notherline\n")
+        if not tools.os_info.is_windows:
+            import subprocess
+            output = subprocess.check_output(["file", fp], stderr=subprocess.STDOUT)
+            self.assertIn("ASCII text", str(output))
+            self.assertNotIn("CRLF", str(output))
+
+            tools.unix2dos(fp)
+            output = subprocess.check_output(["file", fp], stderr=subprocess.STDOUT)
+            self.assertIn("ASCII text", str(output))
+            self.assertIn("CRLF", str(output))
+        else:
+            fc = tools.load(fp)
+            self.assertNotIn("\r\n", fc)
+            tools.unix2dos(fp)
+            fc = tools.load(fp)
+            self.assertIn("\r\n", fc)
+
+        self.assertEquals("a line\r\notherline\r\n", str(tools.load(fp)))
+
+        fp = save_file(b"a line\r\notherline\r\n")
+        if not tools.os_info.is_windows:
+            import subprocess
+            output = subprocess.check_output(["file", fp], stderr=subprocess.STDOUT)
+            self.assertIn("ASCII text", str(output))
+            self.assertIn("CRLF", str(output))
+
+            tools.dos2unix(fp)
+            output = subprocess.check_output(["file", fp], stderr=subprocess.STDOUT)
+            self.assertIn("ASCII text", str(output))
+            self.assertNotIn("CRLF", str(output))
+        else:
+            fc = tools.load(fp)
+            self.assertIn("\r\n", fc)
+            tools.dos2unix(fp)
+            fc = tools.load(fp)
+            self.assertNotIn("\r\n", fc)
+
+        self.assertEquals("a line\notherline\n", str(tools.load(fp)))
+
+    def unix_to_dos_conanfile_test(self):
+        client = TestClient()
+        conanfile = """
+import os
+from conans import ConanFile, tools
+
+class HelloConan(ConanFile):
+    name = "Hello"
+    version = "0.1"
+    exports_sources = "file.txt"
+
+    def build(self):
+        assert("\\r\\n" in tools.load("file.txt"))
+        tools.dos2unix("file.txt")
+        assert("\\r\\n" not in tools.load("file.txt"))
+        tools.unix2dos("file.txt")
+        assert("\\r\\n" in tools.load("file.txt"))
+"""
+        client.save({"conanfile.py": conanfile, "file.txt": "hello\r\n"})
+        client.run("create . user/channel")