Repository: ParkMyCar/compact_str
Pull request #16 (2021-09-28 12:41:03)

Add && Impl new trait ToCompactStr
Similar to [`std::string::ToString`](https://doc.rust-lang.org/std/string/trait.ToString.html), `ToCompactStr` convert any value implements `fmt::Display` into a `CompactStr` efficiently.

However, due to the fact that `Arc::<[MaybeUninit<T>]>` is still unstable, there is no way to implement `HeapString` efficiently without first creating a `String`, then copy it onto the buffer and deallocate the original `String`.

It seems that the compiler may not be able to optimize these calls, especially with `std::format!` calling `format`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

ParkMyCar: Thanks for the PR!

I really like the idea of a `ToCompactStr` trait. But you're right, without the nightly compiler we can't do this very efficiently. The `HeapString` representation will be changing soon, see #12, I plan to spend some time on it this weekend. Let's hold off on landing this until we make the changes to `HeapString` 
NobodyXu: Sure, I will wait for #12 
ParkMyCar: Hey @NobodyXu, now that `CompactStr` is more stable, would you want to take another shot at this?

I think the idea of a `ToCompactStr` trait is great, I'm a bit apprehensive of manually implementing it for any type that implements `Display` though because of performance. Maybe we could define the trait, and then manually implement for a few common types like all the number types (e.g. `u8`, `isize`, `f32`, etc), and then maybe `bool` and `char`? IMO that would go a fairly long way for common use cases, what do you think?

For efficiently converting number types to strings take a look at [`itoa`](https://github.com/dtolnay/itoa) and [`ryu`](https://github.com/dtolnay/ryu)
NobodyXu: > Hey @NobodyXu, now that `CompactStr` is more stable, would you want to take another shot at this?

Yes, sure.

> I think the idea of a `ToCompactStr` trait is great, I'm a bit apprehensive of manually implementing it for any type that implements `Display` though because of performance. Maybe we could define the trait, and then manually implement for a few common types like all the number types (e.g. `u8`, `isize`, `f32`, etc), and then maybe `bool` and `char`? IMO that would go a fairly long way for common use cases, what do you think?

Hmmm, I don't like the idea of implementing only for common types, as it would make `ToCompactStr` less useful and
provide different semantics compared to `ToString`.

For common number types, maybe we can provide a optional specialization for them using
 [castaway](https://crates.io/crates/castaway) and gate it using a feature flag?
ParkMyCar: > Hmmm, I don't like the idea of implementing only for common types, as it would make ToCompactStr less useful and
provide different semantics compared to ToString.
Fair enough, that makes sense to me!

Ugh I wish [specialization](https://rust-lang.github.io/rfcs/1210-impl-specialization.html) was stable. `castaway` looks pretty cool, let's give it a shot!
NobodyXu: > > Hmmm, I don't like the idea of implementing only for common types, as it would make ToCompactStr less useful and
> > provide different semantics compared to ToString.
> 
> Fair enough, that makes sense to me!
> 
> Ugh I wish [specialization](https://rust-lang.github.io/rfcs/1210-impl-specialization.html) was stable. Regardless, `castaway` looks pretty cool, let's give it a shot!

I am quite busy recently, so it might take me some time to implement it.

Patch:
diff --git a/compact_str/Cargo.toml b/compact_str/Cargo.toml
index 2eae622b..6485d5e0 100644
--- a/compact_str/Cargo.toml
+++ b/compact_str/Cargo.toml
@@ -11,12 +11,14 @@ readme = "../README.md"
 keywords = ["string", "compact", "small", "memory", "mutable"]
 categories = ["encoding", "parsing", "memory-management", "text-processing"]
 
-# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
-
 [dependencies]
 bytes = { version = "1", optional = true }
 serde = { version = "1", optional = true }
 
+castaway = "0.2.1"
+itoa = "1"
+ryu = "1"
+
 [dev-dependencies]
 proptest = { version = "1", default-features = false, features = ["std"] }
 test-strategy = "0.1.2"
diff --git a/compact_str/src/lib.rs b/compact_str/src/lib.rs
index c7e24ea5..28e006af 100644
--- a/compact_str/src/lib.rs
+++ b/compact_str/src/lib.rs
@@ -34,9 +34,14 @@ use std::borrow::Cow;
 mod asserts;
 mod features;
 
+mod utility;
+use utility::count;
+
 mod repr;
 use repr::Repr;
 
+mod to_compact_str_specialisation;
+
 #[cfg(test)]
 mod tests;
 
@@ -737,3 +742,55 @@ impl Add<CompactStr> for String {
 }
 
 crate::asserts::assert_size_eq!(CompactStr, String);
+
+/// A trait for converting a value to a `CompactStr`.
+///
+/// This trait is automatically implemented for any type which implements the
+/// [`Display`] trait. As such, `ToCompactStr` shouldn't be implemented directly:
+/// [`Display`] should be implemented instead, and you get the `ToCompactStr`
+/// implementation for free.
+///
+/// [`Display`]: fmt::Display
+pub trait ToCompactStr {
+    /// Converts the given value to a `CompactStr`.
+    ///
+    /// # Examples
+    ///
+    /// Basic usage:
+    ///
+    /// ```
+    /// use compact_str::ToCompactStr;
+    ///
+    /// let i = 5;
+    /// let five = "5".to_compact_str();
+    ///
+    /// assert_eq!(five, i.to_string());
+    /// ```
+    fn to_compact_str(&self) -> CompactStr;
+}
+
+/// # Panics
+///
+/// In this implementation, the `to_compact_str` method panics
+/// if the `Display` implementation returns an error.
+/// This indicates an incorrect `Display` implementation
+/// since `std::fmt::Write for String` never returns an error itself and
+/// the implementation of `ToCompactStr::to_compact_str` only panic if the `Display`
+/// implementation is incorrect..
+impl<T: fmt::Display> ToCompactStr for T {
+    #[inline]
+    fn to_compact_str(&self) -> CompactStr {
+        use fmt::Write;
+
+        if let Some(compact_str) = to_compact_str_specialisation::to_compact_str_specialised(self) {
+            compact_str
+        } else {
+            let num_bytes = count(self);
+
+            let mut compact_str = CompactStr::with_capacity(num_bytes);
+            write!(&mut compact_str, "{}", self).expect("fmt::Display incorrectly implemented!");
+
+            compact_str
+        }
+    }
+}
diff --git a/compact_str/src/repr/mod.rs b/compact_str/src/repr/mod.rs
index d6809ff8..bf64b0ba 100644
--- a/compact_str/src/repr/mod.rs
+++ b/compact_str/src/repr/mod.rs
@@ -21,7 +21,7 @@ use discriminant::{
 use heap::HeapString;
 use inline::InlineString;
 
-const MAX_SIZE: usize = std::mem::size_of::<String>();
+pub(crate) const MAX_SIZE: usize = std::mem::size_of::<String>();
 const EMPTY: Repr = Repr {
     inline: InlineString::new_const(""),
 };
diff --git a/compact_str/src/to_compact_str_specialisation.rs b/compact_str/src/to_compact_str_specialisation.rs
new file mode 100644
index 00000000..3ef0ed3f
--- /dev/null
+++ b/compact_str/src/to_compact_str_specialisation.rs
@@ -0,0 +1,157 @@
+use castaway::{
+    cast,
+    LifetimeFree,
+};
+
+use super::CompactStr;
+
+/// # Safety
+///
+///  - CompactStr does not contain any lifetime
+///  - CompactStr is 'static
+///  - CompactStr is a container to `u8`, which is `LifetimeFree`.
+unsafe impl LifetimeFree for CompactStr {}
+
+const TRUE_COMPACT_STR: CompactStr = CompactStr::new_inline("true");
+const FALSE_COMPACT_STR: CompactStr = CompactStr::new_inline("false");
+
+#[inline(always)]
+pub(super) fn to_compact_str_specialised<T>(val: &T) -> Option<CompactStr> {
+    if let Some(compact_str) = int_spec::to_compact_str_specialised(val) {
+        Some(compact_str)
+    } else if let Some(compact_str) = float_spec::to_compact_str_specialised(val) {
+        Some(compact_str)
+    } else if let Ok(boolean) = cast!(val, &bool) {
+        Some(if *boolean {
+            TRUE_COMPACT_STR
+        } else {
+            FALSE_COMPACT_STR
+        })
+    } else if let Ok(character) = cast!(val, &char) {
+        Some(CompactStr::new_inline(
+            character.encode_utf8(&mut [0; 4][..]),
+        ))
+    } else if let Ok(string) = cast!(val, &String) {
+        Some(CompactStr::new(&*string))
+    } else if let Ok(compact_str) = cast!(val, &CompactStr) {
+        Some(compact_str.clone())
+    } else {
+        None
+    }
+}
+
+mod int_spec {
+    use core::num;
+
+    use itoa::{
+        Buffer,
+        Integer,
+    };
+
+    use super::*;
+    use crate::repr::MAX_SIZE;
+
+    trait IsNewInlineable {
+        const IS_NEW_INLINEABLE: bool;
+    }
+
+    macro_rules! impl_integer_new_inlineable {
+        ($int: ty, $len: expr) => {
+            impl IsNewInlineable for $int {
+                const IS_NEW_INLINEABLE: bool = $len <= MAX_SIZE;
+            }
+        };
+    }
+
+    impl_integer_new_inlineable!(u8, 3);
+    impl_integer_new_inlineable!(i8, 4);
+
+    impl_integer_new_inlineable!(u16, 5);
+    impl_integer_new_inlineable!(i16, 6);
+
+    impl_integer_new_inlineable!(u32, 10);
+    impl_integer_new_inlineable!(i32, 11);
+
+    impl_integer_new_inlineable!(u64, 20);
+    impl_integer_new_inlineable!(i64, 21);
+
+    impl_integer_new_inlineable!(u128, 39);
+    impl_integer_new_inlineable!(i128, 40);
+
+    // For 32-bit and 64-bit arch, usize and isize can be stored inlined.
+    impl IsNewInlineable for usize {
+        const IS_NEW_INLINEABLE: bool = true;
+    }
+    impl IsNewInlineable for isize {
+        const IS_NEW_INLINEABLE: bool = true;
+    }
+
+    fn int_to_compact_str<T: Integer + IsNewInlineable>(int: T) -> CompactStr {
+        let mut buffer = Buffer::new();
+        let s = buffer.format(int);
+
+        if T::IS_NEW_INLINEABLE {
+            CompactStr::new_inline(s)
+        } else {
+            CompactStr::new(s)
+        }
+    }
+
+    macro_rules! specialise {
+        ($val: expr, $int: ty, $nonzero_int: ty) => {
+            if let Ok(int) = cast!($val, &$int) {
+                return Some(int_to_compact_str(*int));
+            } else if let Ok(nonzero_int) = cast!($val, &$nonzero_int) {
+                return Some(int_to_compact_str(nonzero_int.get()));
+            }
+        };
+    }
+
+    #[inline(always)]
+    pub(super) fn to_compact_str_specialised<T>(val: &T) -> Option<CompactStr> {
+        specialise!(val, i8, num::NonZeroI8);
+        specialise!(val, u8, num::NonZeroU8);
+
+        specialise!(val, i16, num::NonZeroI16);
+        specialise!(val, u16, num::NonZeroU16);
+
+        specialise!(val, i32, num::NonZeroI32);
+        specialise!(val, u32, num::NonZeroU32);
+
+        specialise!(val, i64, num::NonZeroI64);
+        specialise!(val, u64, num::NonZeroU64);
+
+        specialise!(val, i128, num::NonZeroI128);
+        specialise!(val, u128, num::NonZeroU128);
+
+        specialise!(val, isize, num::NonZeroIsize);
+        specialise!(val, usize, num::NonZeroUsize);
+
+        None
+    }
+}
+
+mod float_spec {
+    use ryu::{
+        Buffer,
+        Float,
+    };
+
+    use super::*;
+
+    #[inline(always)]
+    fn float_to_compact_str(float: impl Float) -> CompactStr {
+        CompactStr::new(Buffer::new().format(float))
+    }
+
+    #[inline(always)]
+    pub(super) fn to_compact_str_specialised<T>(val: &T) -> Option<CompactStr> {
+        if let Ok(float) = cast!(val, &f32) {
+            Some(float_to_compact_str(*float))
+        } else if let Ok(float) = cast!(val, &f64) {
+            Some(float_to_compact_str(*float))
+        } else {
+            None
+        }
+    }
+}
diff --git a/compact_str/src/utility.rs b/compact_str/src/utility.rs
new file mode 100644
index 00000000..98cd0eb5
--- /dev/null
+++ b/compact_str/src/utility.rs
@@ -0,0 +1,51 @@
+use core::fmt::{
+    self,
+    Write,
+};
+
+#[inline(always)]
+pub(crate) fn count(args: impl fmt::Display) -> usize {
+    let mut sink = Sink(0);
+    write!(&mut sink, "{}", args).unwrap();
+    sink.0
+}
+
+/// A special kind of sink that records the size of bytes writen into it.
+struct Sink(usize);
+
+impl Write for Sink {
+    #[inline(always)]
+    fn write_str(&mut self, s: &str) -> fmt::Result {
+        self.0 += s.len();
+        Ok(())
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use core::format_args;
+
+    use super::count;
+
+    macro_rules! count {
+        ( $fmt:expr $(, $args:tt)* ) => {
+            count(
+                format_args!(
+                    $fmt,
+                    $(
+                        $args,
+                    )*
+                )
+            )
+        };
+    }
+
+    #[test]
+    fn test_count() {
+        assert_eq!(5, count!("{}", "12345"));
+        assert_eq!(6, count!("1{}", "12345"));
+        assert_eq!(7, count!("1{}{}", "12345", 2));
+        assert_eq!(8, count!("1{}{}{}", "12345", 2, '2'));
+        assert_eq!(12, count!("1{}{}{}{}", "12345", 2, '2', 1000));
+    }
+}


Test patch:
diff --git a/compact_str/src/tests.rs b/compact_str/src/tests.rs
index 3689478a..2fea9611 100644
--- a/compact_str/src/tests.rs
+++ b/compact_str/src/tests.rs
@@ -1,10 +1,14 @@
+use std::num;
 use std::str::FromStr;
 
 use proptest::prelude::*;
 use proptest::strategy::Strategy;
 use test_strategy::proptest;
 
-use crate::CompactStr;
+use crate::{
+    CompactStr,
+    ToCompactStr,
+};
 
 #[cfg(target_pointer_width = "64")]
 const MAX_SIZE: usize = 24;
@@ -365,3 +369,87 @@ fn test_plus_operator() {
     assert_eq!(CompactStr::from("a") + String::from("b"), "ab");
     assert_eq!(String::from("a") + CompactStr::from("b"), "ab");
 }
+
+macro_rules! format_compact {
+    ( $fmt:expr $(, $args:tt)* ) => {
+        ToCompactStr::to_compact_str(
+            &core::format_args!(
+                $fmt,
+                $(
+                    $args,
+                )*
+            )
+        )
+    };
+}
+
+macro_rules! assert_int_MAX_to_compact_str {
+    ($int: ty) => {
+        assert_eq!(&*<$int>::MAX.to_string(), &*<$int>::MAX.to_compact_str());
+    };
+}
+
+#[test]
+fn test_to_compact_str() {
+    // Test specialisation for bool, char and String
+    assert_eq!(&*true.to_string(), "true".to_compact_str());
+    assert_eq!(&*false.to_string(), "false".to_compact_str());
+
+    assert_eq!("1", '1'.to_compact_str());
+    assert_eq!("2333", "2333".to_string().to_compact_str());
+    assert_eq!("2333", "2333".to_compact_str().to_compact_str());
+
+    // Test specialisation for int and nonzero_int using itoa
+    assert_eq!("234", 234.to_compact_str());
+    assert_eq!("234", num::NonZeroU64::new(234).unwrap().to_compact_str());
+
+    assert_int_MAX_to_compact_str!(u8);
+    assert_int_MAX_to_compact_str!(i8);
+
+    assert_int_MAX_to_compact_str!(u16);
+    assert_int_MAX_to_compact_str!(i16);
+
+    assert_int_MAX_to_compact_str!(u32);
+    assert_int_MAX_to_compact_str!(i32);
+
+    assert_int_MAX_to_compact_str!(u64);
+    assert_int_MAX_to_compact_str!(i64);
+
+    assert_int_MAX_to_compact_str!(usize);
+    assert_int_MAX_to_compact_str!(isize);
+
+    // Test specialisation for f32 and f64 using ryu
+    assert_eq!(
+        (&*3.2_f32.to_string(), &*288888.290028_f64.to_string()),
+        (
+            &*3.2_f32.to_compact_str(),
+            &*288888.290028_f64.to_compact_str()
+        )
+    );
+
+    assert_eq!("inf", f32::INFINITY.to_compact_str());
+    assert_eq!("-inf", f32::NEG_INFINITY.to_compact_str());
+
+    assert_eq!("inf", f64::INFINITY.to_compact_str());
+    assert_eq!("-inf", f64::NEG_INFINITY.to_compact_str());
+
+    assert_eq!("NaN", f32::NAN.to_compact_str());
+    assert_eq!("NaN", f64::NAN.to_compact_str());
+
+    // Test generic Display implementation
+    assert_eq!("234", "234".to_compact_str());
+    assert_eq!("12345", format_compact!("{}", "12345"));
+    assert_eq!("112345", format_compact!("1{}", "12345"));
+    assert_eq!("1123452", format_compact!("1{}{}", "12345", 2));
+    assert_eq!("11234522", format_compact!("1{}{}{}", "12345", 2, '2'));
+    assert_eq!(
+        "112345221000",
+        format_compact!("1{}{}{}{}", "12345", 2, '2', 1000)
+    );
+
+    // Test string longer than repr::MAX_SIZE
+    assert_eq!(
+        "01234567890123456789999999",
+        format_compact!("0{}67890123456789{}", "12345", 999999)
+    );
+}