Repository: uutils/coreutils (Rust)

Issue:
`expr` is failing with multibyte chars
It causes https://github.com/coreutils/coreutils/blob/master/tests/misc/expr-multibyte.pl to fail
```
$ ./target/debug/coreutils expr length αbcdef
7
```

GNU:
```
$ expr length αbcdef
6
```
needs to have a different locale compiled like 
```
sudo locale-gen fr_FR.UTF-8
```


Discussion:
Of course, it is about rust. See https://doc.rust-lang.org/book/ch08-02-strings.html#internal-representation

Simple testcase:
```
fn main() {
    let s = String::from("αbcdef");
    assert_eq!(s.len(), 6);
}
```
=> 
```
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `7`,
 right: `6`', src/main.rs:3:5
```
I did some extra testing to check whether we need unicode segmentation here and we don't. GNU `expr` outputs a length of 2 for this emoji:
```
[src/main.rs:4] "🇳🇱".len() = 8
[src/main.rs:5] "🇳🇱".chars().count() = 2
[src/main.rs:6] UnicodeSegmentation::graphemes("🇳🇱", true).count() = 1
```
[Playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=f397b963203a431c6378012433641794)
Yeah, I am working on a fix :)

To reproduce:
bash util/run-gnu-test.sh  tests/misc/expr-multibyte                        


Actually, my patch was wrong, it should take in account the locale
```
$ LANG=C expr length αbcdef
7
$ LANG=fr_FR.UTF-8 expr length αbcdef
6
```

seems that we should use MB_CUR_MAX to see the number of bytes
Hi all, is this still an issue?

Chris
Closing this issue, it looks like the issue has been fixed in the meantime and the GNU test `tests/expr/expr-multibyte.pl` passes.



Patch:
diff --git a/src/uu/expr/src/syntax_tree.rs b/src/uu/expr/src/syntax_tree.rs
index c11689a07f3..4dafffbf0f8 100644
--- a/src/uu/expr/src/syntax_tree.rs
+++ b/src/uu/expr/src/syntax_tree.rs
@@ -10,7 +10,7 @@
 //! * `<https://en.wikipedia.org/wiki/Shunting-yard_algorithm>`
 //!
 
-// spell-checker:ignore (ToDO) binop binops ints paren prec
+// spell-checker:ignore (ToDO) binop binops ints paren prec multibytes
 
 use num_bigint::BigInt;
 use num_traits::{One, Zero};
@@ -465,7 +465,9 @@ fn operator_match(values: &[String]) -> Result<String, String> {
 
 fn prefix_operator_length(values: &[String]) -> String {
     assert!(values.len() == 1);
-    values[0].len().to_string()
+    // Use chars().count() as we can have some multibytes chars
+    // See https://github.com/uutils/coreutils/issues/3132
+    values[0].chars().count().to_string()
 }
 
 fn prefix_operator_index(values: &[String]) -> String {


Test patch:
diff --git a/tests/by-util/test_expr.rs b/tests/by-util/test_expr.rs
index 30e3016a313..3a753aa1bcc 100644
--- a/tests/by-util/test_expr.rs
+++ b/tests/by-util/test_expr.rs
@@ -1,3 +1,5 @@
+// spell-checker:ignore αbcdef
+
 use crate::common::util::*;
 
 #[test]
@@ -95,6 +97,27 @@ fn test_and() {
     new_ucmd!().args(&["", "&", "1"]).run().stdout_is("0\n");
 }
 
+#[test]
+fn test_length_fail() {
+    new_ucmd!().args(&["length", "αbcdef", "1"]).fails();
+}
+
+#[test]
+fn test_length() {
+    new_ucmd!()
+        .args(&["length", "abcdef"])
+        .succeeds()
+        .stdout_only("6\n");
+}
+
+#[test]
+fn test_length_mb() {
+    new_ucmd!()
+        .args(&["length", "αbcdef"])
+        .succeeds()
+        .stdout_only("6\n");
+}
+
 #[test]
 fn test_substr() {
     new_ucmd!()