Repository: jorgecarleitao/arrow2
Pull request #1128 (2022-06-29 16:28:00)

Removed `O(N)` clone in writing CSV
This is only applicable to the `serialize` function.

codecov[bot]: # [Codecov](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao) Report
> Merging [#1128](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao) (1b785ff) into [main](https://codecov.io/gh/jorgecarleitao/arrow2/commit/81ab424913daad775ce63be62f7b57b145313267?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao) (81ab424) will **increase** coverage by `0.07%`.
> The diff coverage is `100.00%`.

```diff
@@            Coverage Diff             @@
##             main    #1128      +/-   ##
==========================================
+ Coverage   83.37%   83.45%   +0.07%     
==========================================
  Files         367      367              
  Lines       35574    35573       -1     
==========================================
+ Hits        29661    29687      +26     
+ Misses       5913     5886      -27     
```


| [Impacted Files](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao) | Coverage Δ | |
|---|---|---|
| [src/io/csv/write/mod.rs](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao#diff-c3JjL2lvL2Nzdi93cml0ZS9tb2QucnM=) | `100.00% <100.00%> (+34.93%)` | :arrow_up: |
| [src/array/binary/mod.rs](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao#diff-c3JjL2FycmF5L2JpbmFyeS9tb2QucnM=) | `88.88% <0.00%> (-1.24%)` | :arrow_down: |
| [src/io/ipc/read/reader.rs](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao#diff-c3JjL2lvL2lwYy9yZWFkL3JlYWRlci5ycw==) | `96.02% <0.00%> (ø)` | |
| [src/bitmap/immutable.rs](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao#diff-c3JjL2JpdG1hcC9pbW11dGFibGUucnM=) | `81.48% <0.00%> (+0.61%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao). Last update [81ab424...1b785ff](https://codecov.io/gh/jorgecarleitao/arrow2/pull/1128?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao).


Patch:
diff --git a/src/io/csv/write/mod.rs b/src/io/csv/write/mod.rs
index 1f124ef676..62f228ef14 100644
--- a/src/io/csv/write/mod.rs
+++ b/src/io/csv/write/mod.rs
@@ -27,16 +27,16 @@ fn new_serializers<'a, A: AsRef<dyn Array>>(
 /// The vector is guaranteed to have `columns.len()` entries.
 /// Each `row` is guaranteed to have `columns.array().len()` fields.
 pub fn serialize<A: AsRef<dyn Array>>(
-    columns: &Chunk<A>,
+    chunk: &Chunk<A>,
     options: &SerializeOptions,
 ) -> Result<Vec<Vec<u8>>> {
-    let mut serializers = new_serializers(columns, options)?;
+    let mut serializers = new_serializers(chunk, options)?;
 
-    let mut rows = Vec::with_capacity(columns.len());
+    let mut rows = Vec::with_capacity(chunk.len());
     let mut row = vec![];
 
     // this is where the (expensive) transposition happens: the outer loop is on rows, the inner on columns
-    (0..columns.len()).try_for_each(|_| {
+    (0..chunk.len()).try_for_each(|_| {
         serializers
             .iter_mut()
             // `unwrap` is infalible because `array.len()` equals `Chunk::len`
@@ -49,8 +49,7 @@ pub fn serialize<A: AsRef<dyn Array>>(
             // replace last delimiter with new line
             let last_byte = row.len() - 1;
             row[last_byte] = b'\n';
-            rows.push(row.clone());
-            row.clear();
+            rows.push(std::mem::take(&mut row));
         }
         Result::Ok(())
     })?;


Test patch:
diff --git a/tests/it/io/csv/write.rs b/tests/it/io/csv/write.rs
index 88b502adfb..44bec50497 100644
--- a/tests/it/io/csv/write.rs
+++ b/tests/it/io/csv/write.rs
@@ -423,3 +423,23 @@ fn write_escaping_resize_local_buf() {
         test_generic(chunk,  &format!("\"{}\"\n", payload));
     }
 }
+
+#[test]
+fn serialize_vec() -> Result<()> {
+    let columns = data();
+
+    let options = SerializeOptions::default();
+
+    let data = serialize(&columns, &options)?;
+
+    // check
+    assert_eq!(
+        vec![
+            b"a b,123.564532,3,true,,00:20:34,d\n".to_vec(),
+            b"c,,2,false,2019-04-18 10:54:47.378,06:51:20,a b\n".to_vec(),
+            b"d,-556132.25,1,,2019-04-18 02:45:55.555,23:46:03,c\n".to_vec(),
+        ],
+        data
+    );
+    Ok(())
+}