Repository: crytic/slither

Issue:
Incorrect source mappings because of certain (Unicode?) characters in comments
Certain characters (or scripts) in Solidity comments appear to cause incorrect source mappings.

For example, in `0x06012c8cf97bead5deae237070f9587f8e7a266d_KittyCore.sol`, the symbol that looks like underscore in "email_protected":
```
/// @author Dieter Shirley <<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="6004051405200118090f0d1a050e4e030f">[email_protected]</a>> (https://github.com/dete) 
```
Similarly, the Asian characters in below comments from `0x5d0d76787d9d564061dd23f8209f804a3b8ad2f2_FoMo3Dlong.sol` also cause source mapping problems:

```
struct Round {
        uint256 plyr;   // pID of player in lead， lead领导吗？                                                                
        uint256 team;   // tID of team in lead                                                                                 
        uint256 end;    // time ends/ended                                                                                     
        bool ended;     // has round end function been ran  这个开关值得研究下                                                 
	uint256 strt;   // time round started                                                                                  
	uint256 keys;   // keys                                                                                                
	uint256 eth;    // total eth in                                                                                        
	uint256 pot;    // eth to pot (during round) / final amount paid to winner (after round ends)                          
        uint256 mask;   // global mask                                                                                         
        uint256 ico;    // total eth sent in during ICO phase                                                                  
        uint256 icoGen; // total eth for gen during ICO phase                                                                  
        uint256 icoAvg; // average key price for ICO phase                                                                     
    }
```


Discussion:

Resolving patch:
diff --git a/slither/core/source_mapping/source_mapping.py b/slither/core/source_mapping/source_mapping.py
--- a/slither/core/source_mapping/source_mapping.py
+++ b/slither/core/source_mapping/source_mapping.py
@@ -20,6 +20,7 @@ def _compute_line(source_code, start, length):
 
             Not done in an efficient way
         """
+        source_code = source_code.encode('utf-8')
         total_length = len(source_code)
         source_code = source_code.splitlines(True)
         counter = 0
@@ -29,17 +30,18 @@ def _compute_line(source_code, start, length):
         ending_column = None
         while counter < total_length:
             # Determine the length of the line, and advance the line number
-            lineLength = len(source_code[i])
+            line_content = source_code[i]
+            line_length = len(line_content)
             i = i + 1
 
             # Determine our column numbers.
-            if starting_column is None and counter + lineLength > start:
+            if starting_column is None and counter + line_length > start:
                 starting_column = (start - counter) + 1
-            if starting_column is not None and ending_column is None and counter + lineLength > start + length:
+            if starting_column is not None and ending_column is None and counter + line_length > start + length:
                 ending_column = ((start + length) - counter) + 1
 
             # Advance the current position counter, and determine line numbers.
-            counter += lineLength
+            counter += line_length
             if counter > start:
                 lines.append(i)
 


Test patch:
diff --git a/tests/source_mapping.sol b/tests/source_mapping.sol
new file mode 100644
--- /dev/null
+++ b/tests/source_mapping.sol
@@ -0,0 +1,22 @@
+contract A{
+
+    // ëëëëëëëëëëëëë这这这这这这这这这这
+
+    address unused;
+
+
+    address unused2;
+
+
+    // ই এই এই এইই এই এই এইই এই এই এই
+
+    address unused3;
+
+
+    address unused4;
+
+    // 这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这这
+    address used;
+
+
+}