Repository: capnproto/capnproto-rust (language: rust)

Issue:
Using CodeGeneratorRequest directly
Is there any way to access the `CodeGeneratorRequest` object from what was generated in the `build.rs`? What I have in mind is a `proc_macro` that implements additional traits for generated Reader and Builder structs based on custom annotations in the `.capnp` file?

If there is no way for that, can I write a simple PR that adds a method to `capnpc::CompilerCommand` that is similar to [`file_descriptor_set_path`](https://docs.rs/prost-build/latest/prost_build/struct.Config.html#method.file_descriptor_set_path) from `prost_build` to output a serialized `CodeGeneratorRequest` into the `OUT_DIR` so I can read it from my `proc_macro`? Or maybe some different way to access it like returning a `CodeGeneratorRequest` reader from a separate version of `CompilerCommand::run()` function?


Discussion / hints:
Can you describe more about what your additional traits will do?

capnproto-c++ saves essentially all of the infomation of the `CodeGeneratorRequest` into the generated source code itself, in order to support the [dynamic reflection API](https://capnproto.org/cxx.html#dynamic-reflection). Would something like that help with what you're doing?
I'm trying to make a structured KV database with schema based on Capnp schema. I did something similar with Protobuf already, but Protos are a bit awkward about custom annotations so I wanted to try to do the same with Capnp instead. My idea was to read a proto file descriptor, find annotations that tell us how to deduce a key from a value: https://github.com/freopen/pbdb/blob/main/integration-tests/proto/tests.proto#L8, then extend the type with methods to interact with database: https://github.com/freopen/pbdb/blob/main/src/lib.rs#L21, using proc_macros: https://github.com/freopen/pbdb/blob/main/pbdb-macros/src/lib.rs.

You could make const functions returning metadata about the structure, but I believe my proc_macro crate won't be able to use them as it's going to expand into the same space. Using proc derive macros seems complicated because there are no actual fields to attach the macro annotations to, just a set of functions per field, so it will be awkward to use. Maybe having a big fat macro annotation with all the dependencies attached for every Reader and Builder will do but it seems awkward too. Runtime implementation seem to require going through each field's annotations every time we are extracting key from value, sounds much worse than compile-time variant. 
Adding a new option to `CompilerCommand` for saving the `CodeGeneratorRequest` sounds reasonable to me, if that's going to unblock your progress. Working with the raw `CodeGeneratorRequest` is going to be rough, though. It would be nice if `capnpc` exposed a nice API to help with that -- for things like resolving IDs and generics. capnproto-c++ has the `Schema` class for this purpose: https://github.com/capnproto/capnproto/blob/2ed8664a08e2c92f0af39e213c20b13d15c03346/c%2B%2B/src/capnp/schema.h#L77.

Fix patch:
diff --git a/capnpc/src/codegen.rs b/capnpc/src/codegen.rs
index 07951e07f..84028380e 100644
--- a/capnpc/src/codegen.rs
+++ b/capnpc/src/codegen.rs
@@ -36,6 +36,7 @@ use self::FormattedText::{Indent, Line, Branch, BlankLine};
 pub struct CodeGenerationCommand {
     output_directory: PathBuf,
     default_parent_module: Vec<String>,
+    raw_code_generator_request_path: Option<PathBuf>,
 }
 
 impl CodeGenerationCommand {
@@ -44,6 +45,7 @@ impl CodeGenerationCommand {
         CodeGenerationCommand {
             output_directory: PathBuf::new(),
             default_parent_module: Vec::new(),
+            raw_code_generator_request_path: None,
         }
     }
 
@@ -65,6 +67,14 @@ impl CodeGenerationCommand {
         self
     }
 
+    /// Sets the raw code generator request output path.
+    pub fn raw_code_generator_request_path<P>(&mut self, path: P) -> &mut Self
+        where P: AsRef<Path>
+    {
+        self.raw_code_generator_request_path = Some(path.as_ref().to_path_buf());
+        self
+    }
+
     /// Generates Rust code according to a `schema_capnp::code_generator_request` read from `inp`.
     pub fn run<T>(&mut self, inp: T) -> ::capnp::Result<()>
         where T: std::io::Read
@@ -119,6 +129,12 @@ impl CodeGenerationCommand {
                 }
             }
         }
+
+        if let Some(raw_code_generator_request) = &self.raw_code_generator_request_path {
+            let raw_code_generator_request_file = ::std::fs::File::create(&raw_code_generator_request).map_err(convert_io_err)?;
+            serialize::write_message_segments(WriteWrapper{ inner: raw_code_generator_request_file }, &message.into_segments())?;
+        }
+
         Ok(())
     }
 }
@@ -2023,6 +2039,19 @@ impl <R> capnp::io::Read for ReadWrapper<R> where R: std::io::Read {
     }
 }
 
+// The capnp crate defines a blanket impl of capnp::Write for R where R: std::io::Write,
+// but we can't use that here because it lives behind the "std" feature flag.
+struct WriteWrapper<W> where W: std::io::Write {
+    inner: W,
+}
+
+impl <W> capnp::io::Write for WriteWrapper<W> where W: std::io::Write {
+    fn write_all(&mut self, buf: &[u8]) -> ::capnp::Result<()> {
+        std::io::Write::write_all(&mut self.inner, buf).map_err(convert_io_err)?;
+        Ok(())
+    }
+}
+
 #[deprecated(since = "0.14.2", note = "Use CodeGenerationCommand::run() instead.")]
 pub fn generate_code<T>(inp: T, out_dir: &::std::path::Path) -> ::capnp::Result<()>
     where T: ::std::io::Read
diff --git a/capnpc/src/lib.rs b/capnpc/src/lib.rs
index 54d6dd523..e8355e769 100644
--- a/capnpc/src/lib.rs
+++ b/capnpc/src/lib.rs
@@ -105,6 +105,7 @@ pub struct CompilerCommand {
     executable_path: Option<PathBuf>,
     output_path: Option<PathBuf>,
     default_parent_module: Vec<String>,
+    raw_code_generator_request_path: Option<PathBuf>,
 }
 
 impl CompilerCommand {
@@ -118,6 +119,7 @@ impl CompilerCommand {
             executable_path: None,
             output_path: None,
             default_parent_module: Vec::new(),
+            raw_code_generator_request_path: None,
         }
     }
 
@@ -200,6 +202,16 @@ impl CompilerCommand {
         self
     }
 
+    /// If set, the generator will also write a file containing the raw code generator request to the
+    /// specified path.
+    pub fn raw_code_generator_request_path<P>(&mut self, path: P) -> &mut CompilerCommand
+    where
+        P: AsRef<Path>,
+    {
+        self.raw_code_generator_request_path = Some(path.as_ref().to_path_buf());
+        self
+    }
+
     /// Runs the command.
     /// Returns an error if `OUT_DIR` or a custom output directory was not set, or if `capnp compile` fails.
     pub fn run(&mut self) -> ::capnp::Result<()> {
@@ -263,6 +275,9 @@ impl CompilerCommand {
         code_generation_command
             .output_directory(output_path)
             .default_parent_module(self.default_parent_module.clone());
+        if let Some(raw_code_generator_request_path) = &self.raw_code_generator_request_path {
+            code_generation_command.raw_code_generator_request_path(raw_code_generator_request_path.clone());
+        }
 
         run_command(command, code_generation_command).map_err(|error| {
             ::capnp::Error::failed(format!(


Test patch:
diff --git a/capnpc/test/Cargo.toml b/capnpc/test/Cargo.toml
index 758d22d92..9487cfb55 100644
--- a/capnpc/test/Cargo.toml
+++ b/capnpc/test/Cargo.toml
@@ -15,3 +15,4 @@ capnpc = { path = "../" }
 
 [dependencies]
 capnp = { path = "../../capnp" }
+capnpc = { path = "../" }
diff --git a/capnpc/test/build.rs b/capnpc/test/build.rs
index 0dac93a0b..8a5a007f0 100644
--- a/capnpc/test/build.rs
+++ b/capnpc/test/build.rs
@@ -6,6 +6,10 @@ fn main() {
         .file("schema/test-in-dir.capnp")
         .file("schema-with-src-prefix/test-in-src-prefix-dir.capnp")
         .src_prefix("schema-with-src-prefix")
+        .raw_code_generator_request_path(
+            std::env::var("OUT_DIR").expect("OUT_DIR env var is not set")
+                + "/raw_code_gen_request.bin",
+        )
         .run()
         .expect("compiling schema");
 
diff --git a/capnpc/test/test.rs b/capnpc/test/test.rs
index fad2a5e8c..61b1a81d8 100644
--- a/capnpc/test/test.rs
+++ b/capnpc/test/test.rs
@@ -1588,4 +1588,20 @@ mod tests {
         let message_reader = TypedReader::<_, test_all_types::Owned>::new(reader);
         ::test_util::CheckTestMessage::check_test_message(message_reader.get().unwrap());
     }
+
+    #[test] 
+    fn test_raw_code_generator_request_path() {
+        use std::fs;
+        use capnp::serialize;
+
+        let raw_code_gen_request = fs::read(
+            std::env::var("OUT_DIR").expect("OUT_DIR env var is not set")
+                + "/raw_code_gen_request.bin"
+        ).expect("Failed to open raw code gen request file");
+
+        let reader = serialize::read_message(raw_code_gen_request.as_slice(), ReaderOptions::new()).unwrap();
+        let generator_context = capnpc::codegen::GeneratorContext::new(&reader).unwrap();
+        assert!(generator_context.node_map.len() > 0);
+        assert!(generator_context.scope_map.len() > 0);
+    }
 }