Repository: RustPython/RustPython (language: rust)

Issue:
[RFC] A fake module from source API (mostly for wasm targets)
## Summary

Provides an API `VirtualMachine::compile_module` that creates a Python module from source code.

<!-- A quick overview of your suggestion -->

## Detailed Explanation
We can use `vm.compile` and `builtins.exec` to provide dynamic libraries without system IO.

Below is not actual code, but it shows the general flow.
```rust
let scope = rustpython_vm::scope::Scope::with_builtins(None, vm.new_dict());
let code = vm.compile(<source text>, "<source>", Mode::Exec); // compiled code
vm.run_code_obj(code_obj, scope);
// now `scope` contains module attributes
let module = PyModule::new();
for (name, value) in scope.locals {
    module.set_attr(name, value);
}
```

We can package these complex procedures into a single function.


The WASM architecture limits the use of Python modules due to access to system io.
Currently, the only way to use Python modules in WASM is to freeze them at compile time.
This API is not only limited to WASM, but will be very useful for WASM.

<!-- Elaborate on your suggestion in all its details -->

## Drawbacks, Rationale, and Alternatives

<!-- What drawbacks might this solution have? Why do you feel it is necessary? What other options might there be to solving this problem? -->

- Better API name or space? I didn't consider putting `import` in the name, like `vm.import_from_source`, because in Python, `import` implies complex filesystem usage via `importlib`.

## Unresolved Questions

<!-- What would you like feedback on for fleshing out your suggestion? -->



Discussion / hints:
@DimitrisJim @coolreader18 Do you have advices?
Not relevant to naming, but I thought the standard procedure was to make the scope's locals be the actual dict of the module itself?
Ok, I thought there was some sort of function like this already, found it:

https://github.com/RustPython/RustPython/blob/bdb0c8f64557e0822f0bcfd63defbad54625c17a/wasm/lib/src/vm_class.rs#L274-L309
Not totally sure if that's the right naming, but it's prior art at least. Though inject_module does feel good I think, inject isn't overloaded with much in Python so that could be a good choice
Oh though maybe your proposal wouldn't add it to sys.modules? That feels like the actual "injection" in the operation inject_module performs.
Thank you! `inject_module` looks a really good reference. I thought that approach cannot prevent user to put mismatching source code with the same name. But wasm may really need it.
Yes. Context: we can have python sources with custom functions and call them with arguments, instead of running them. Currently we have to import the python source file. Some applications wouldn't want or even have permission to write to files.

Fix patch:
diff --git a/vm/src/import.rs b/vm/src/import.rs
index e3ffbbddbc..f14ae31ef4 100644
--- a/vm/src/import.rs
+++ b/vm/src/import.rs
@@ -106,19 +106,32 @@ pub fn import_file(
     vm: &VirtualMachine,
     module_name: &str,
     file_path: String,
-    content: String,
+    content: &str,
 ) -> PyResult {
     let code = vm
         .compile_with_opts(
-            &content,
+            content,
             crate::compiler::Mode::Exec,
             file_path,
             vm.compile_opts(),
         )
-        .map_err(|err| vm.new_syntax_error(&err, Some(&content)))?;
+        .map_err(|err| vm.new_syntax_error(&err, Some(content)))?;
     import_codeobj(vm, module_name, code, true)
 }
 
+#[cfg(feature = "rustpython-compiler")]
+pub fn import_source(vm: &VirtualMachine, module_name: &str, content: &str) -> PyResult {
+    let code = vm
+        .compile_with_opts(
+            content,
+            crate::compiler::Mode::Exec,
+            "<source>".to_owned(),
+            vm.compile_opts(),
+        )
+        .map_err(|err| vm.new_syntax_error(&err, Some(content)))?;
+    import_codeobj(vm, module_name, code, false)
+}
+
 pub fn import_codeobj(
     vm: &VirtualMachine,
     module_name: &str,
diff --git a/vm/src/vm/interpreter.rs b/vm/src/vm/interpreter.rs
index 4213d81ba9..515a93bfb6 100644
--- a/vm/src/vm/interpreter.rs
+++ b/vm/src/vm/interpreter.rs
@@ -1,9 +1,5 @@
 use super::{setting::Settings, thread, Context, VirtualMachine};
-use crate::{
-    stdlib::{atexit, sys},
-    vm::PyBaseExceptionRef,
-    PyResult,
-};
+use crate::{stdlib::atexit, vm::PyBaseExceptionRef, PyResult};
 use std::sync::atomic::Ordering;
 
 /// The general interface for the VM
@@ -120,7 +116,7 @@ impl Interpreter {
     /// Note that calling `finalize` is not necessary by purpose though.
     pub fn finalize(self, exc: Option<PyBaseExceptionRef>) -> u8 {
         self.enter(|vm| {
-            flush_std(vm);
+            vm.flush_std();
 
             // See if any exception leaked out:
             let exit_code = if let Some(exc) = exc {
@@ -133,22 +129,13 @@ impl Interpreter {
 
             vm.state.finalizing.store(true, Ordering::Release);
 
-            flush_std(vm);
+            vm.flush_std();
 
             exit_code
         })
     }
 }
 
-pub(crate) fn flush_std(vm: &VirtualMachine) {
-    if let Ok(stdout) = sys::get_stdout(vm) {
-        let _ = vm.call_method(&stdout, identifier!(vm, flush).as_str(), ());
-    }
-    if let Ok(stderr) = sys::get_stderr(vm) {
-        let _ = vm.call_method(&stderr, identifier!(vm, flush).as_str(), ());
-    }
-}
-
 #[cfg(test)]
 mod tests {
     use super::*;
diff --git a/vm/src/vm/mod.rs b/vm/src/vm/mod.rs
index bd44eaed38..3588dd2af5 100644
--- a/vm/src/vm/mod.rs
+++ b/vm/src/vm/mod.rs
@@ -517,6 +517,7 @@ impl VirtualMachine {
     /// Roughly equivalent to `import module_name` or `import top.submodule`.
     ///
     /// See also [`import_from`] for more advanced import.
+    /// See also [`rustpython_vm::import::import_source`] and other primitive import functions.
     #[inline]
     pub fn import<'a>(&self, module_name: impl AsPyStr<'a>, level: usize) -> PyResult {
         let module_name = module_name.as_pystr(&self.ctx);
@@ -830,7 +831,7 @@ impl VirtualMachine {
                     );
                     let result = unsafe { sigaction(SIGINT, &action) };
                     if result.is_ok() {
-                        interpreter::flush_std(self);
+                        self.flush_std();
                         kill(getpid(), SIGINT).expect("Expect to be killed.");
                     }
 
diff --git a/vm/src/vm/vm_object.rs b/vm/src/vm/vm_object.rs
index a957ed66ca..8f0f158553 100644
--- a/vm/src/vm/vm_object.rs
+++ b/vm/src/vm/vm_object.rs
@@ -4,6 +4,7 @@ use crate::{
     function::IntoFuncArgs,
     identifier,
     object::{AsObject, PyObject, PyObjectRef, PyResult},
+    stdlib::sys,
     vm::VirtualMachine,
 };
 
@@ -14,15 +15,9 @@ impl VirtualMachine {
     fn _py_panic_failed(&self, exc: PyBaseExceptionRef, msg: &str) -> ! {
         #[cfg(not(all(target_arch = "wasm32", not(target_os = "wasi"))))]
         {
-            let show_backtrace =
-                std::env::var_os("RUST_BACKTRACE").map_or(cfg!(target_os = "wasi"), |v| &v != "0");
-            let after = if show_backtrace {
-                self.print_exception(exc);
-                "exception backtrace above"
-            } else {
-                "run with RUST_BACKTRACE=1 to see Python backtrace"
-            };
-            panic!("{msg}; {after}")
+            self.print_exception(exc);
+            self.flush_std();
+            panic!("{msg}")
         }
         #[cfg(all(target_arch = "wasm32", not(target_os = "wasi")))]
         {
@@ -39,6 +34,16 @@ impl VirtualMachine {
         }
     }
 
+    pub(crate) fn flush_std(&self) {
+        let vm = self;
+        if let Ok(stdout) = sys::get_stdout(vm) {
+            let _ = vm.call_method(&stdout, identifier!(vm, flush).as_str(), ());
+        }
+        if let Ok(stderr) = sys::get_stderr(vm) {
+            let _ = vm.call_method(&stderr, identifier!(vm, flush).as_str(), ());
+        }
+    }
+
     #[track_caller]
     pub fn unwrap_pyresult<T>(&self, result: PyResult<T>) -> T {
         match result {
diff --git a/wasm/lib/src/browser_module.rs b/wasm/lib/src/browser_module.rs
index 32e7877a53..4e5704dc24 100644
--- a/wasm/lib/src/browser_module.rs
+++ b/wasm/lib/src/browser_module.rs
@@ -11,7 +11,7 @@ mod _browser {
         class::PyClassImpl,
         convert::ToPyObject,
         function::{ArgCallable, OptionalArg},
-        import::import_file,
+        import::import_source,
         PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
     };
     use wasm_bindgen::{prelude::*, JsCast};
@@ -245,7 +245,7 @@ mod _browser {
                 .expect("that the vm is valid when the promise resolves");
             stored_vm.interp.enter(move |vm| {
                 let resp_text = text.as_string().unwrap();
-                let res = import_file(vm, module.as_str(), "WEB".to_owned(), resp_text);
+                let res = import_source(vm, module.as_str(), &resp_text);
                 match res {
                     Ok(_) => Ok(JsValue::null()),
                     Err(err) => Err(convert::py_err_to_js_err(vm, &err)),


Test patch: