internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: sbi-dev/sbi

Issue:
check error and warning handling for z_scoring
make sure we issue meaningful warnings depending on the `z_score_x` passed, e.g.. 


Discussion:

Resolving patch:
diff --git a/sbi/utils/sbiutils.py b/sbi/utils/sbiutils.py
--- a/sbi/utils/sbiutils.py
+++ b/sbi/utils/sbiutils.py
@@ -26,7 +26,7 @@ def warn_if_zscoring_changes_data(x: Tensor, duplicate_tolerance: float = 0.1) -
     """Raise warning if z-scoring would create duplicate data points.
 
     Args:
-        x: Simulation outputs.
+        x: Simulated data.
         duplicate_tolerance: Tolerated proportion of duplicates after z-scoring.
     """
 
@@ -36,8 +36,9 @@ def warn_if_zscoring_changes_data(x: Tensor, duplicate_tolerance: float = 0.1) -
     # Check we do have different data in the batch
     if num_unique == 1:
         warnings.warn(
-            """Variance along batches is 0. Therefore, Z-score could not be computed and will be skipped.
-            Check that data is different along all dimensions.""",
+            """Beware that there is only a single unique element in the simulated data.
+            If this is intended, make sure to set `z_score_x='none'` as z-scoring would
+            result in NaNs""",
             UserWarning,
         )
 
@@ -53,12 +54,12 @@ def warn_if_zscoring_changes_data(x: Tensor, duplicate_tolerance: float = 0.1) -
         if num_unique_z < num_unique * (1 - duplicate_tolerance):
             warnings.warn(
                 """Z-scoring these simulation outputs resulted in {num_unique_z} unique
-                datapoints. Before z-scoring, it had been {num_unique}. This can occur due
-                to numerical inaccuracies when the data covers a large range of values.
-                Consider either setting `z_score_x=False` (but beware that this can be
-                problematic for training the NN) or exclude outliers from your dataset.
-                Note: if you have already set `z_score_x=False`, this warning will still be
-                displayed, but you can ignore it.""",
+                datapoints. Before z-scoring, it had been {num_unique}. This can occur 
+                due to numerical inaccuracies when the data covers a large range of 
+                values. Consider either setting `z_score_x=False` (but beware that this 
+                can be problematic for training the NN) or exclude outliers from your 
+                dataset. Note: if you have already set `z_score_x=False`, this warning
+                will still be displayed, but you can ignore it.""",
                 UserWarning,
             )
 


Test patch:
Repository: sbi-dev/sbi

Issue:
SBC fails with embedding nets if x.ndim > 1.
in the SBC loop over `theta` and `x`, `x` needs a batch dimension to avoid `ValueError` in the `NPE` posterior. 

I will make a PR soon. 


Discussion:

Resolving patch:
diff --git a/sbi/analysis/sbc.py b/sbi/analysis/sbc.py
--- a/sbi/analysis/sbc.py
+++ b/sbi/analysis/sbc.py
@@ -169,7 +169,9 @@ def sbc_on_batch(
     dap_samples = torch.zeros_like(thetas)
     ranks = torch.zeros((thetas.shape[0], len(reduce_fns)))
 
-    for idx, (tho, xo) in enumerate(zip(thetas, xs)):
+    for idx in range(thetas.shape[0]):
+        # unsqueeze for potential higher-dimensional data.
+        xo = xs[idx].unsqueeze(0)
         # VI posterior needs to be trained on the current xo.
         if isinstance(posterior, VIPosterior):
             posterior.set_default_x(xo)
@@ -184,7 +186,9 @@ def sbc_on_batch(
         # rank for each posterior dimension as in Talts et al. section 4.1.
         for i, reduce_fn in enumerate(reduce_fns):
             ranks[idx, i] = (
-                (reduce_fn(ths, xo) < reduce_fn(tho.unsqueeze(0), xo)).sum().item()
+                (reduce_fn(ths, xo) < reduce_fn(thetas[idx].unsqueeze(0), xo))
+                .sum()
+                .item()
             )
 
     return ranks, dap_samples


Test patch:
Repository: sbi-dev/sbi

Issue:
links not working on getting started page
https://sbi-dev.github.io/sbi/tutorial/00_getting_started/

broken link: https://www.mackelab.org/sbi/tutorial/02_flexible_interface/

possible solution: links not updated to new website? 


Discussion:
Good catch. I did try to update all links in the docs, so I am hopeful that this is an outlier.

Resolving patch:
diff --git a/sbi/inference/base.py b/sbi/inference/base.py
--- a/sbi/inference/base.py
+++ b/sbi/inference/base.py
@@ -40,7 +40,7 @@ def infer(
     The scope of this function is limited to the most essential features of sbi. For
     more flexibility (e.g. multi-round inference, different density estimators) please
     use the flexible interface described here:
-    https://www.mackelab.org/sbi/tutorial/02_flexible_interface/
+    https://sbi-dev.github.io/sbi/tutorial/02_flexible_interface/
 
     Args:
         simulator: A function that takes parameters $\theta$ and maps them to


Test patch:
Repository: sbi-dev/sbi

Issue:
Add kwargs to SNPE_A.build_posterior to match base class API
The `SNPE_A.build_posterior(...)` method overrides that of the base class to correct for the proposal posterior. 

**Problem:** As a side effect, it has a different signature than the base class method, e.g., it is not possible to pass things like `sample_with`, `mcmc_method`, `mcmc_parameters`. Although it is unlikely that such `kwargs` will be relevant when using `SNPE_A`, it would still be nice to keep the same API as the base class. 

**Solution**: Add a `**kwargs` to `SNPE_A.build_posterior(...)` and then do 

```python
self._posterior = super.build_posterior(
            posterior_estimator=wrapped_density_estimator,  # type: ignore
            prior=prior,
            **kwargs,
        )
```

instead of building the posterior directly with `DirectPosterior(...)`. 


Discussion:
Happy to take care of it.
I also like to work on this if available.
So I think @zinaStef  already modified the arguments, does the method need to provide functions in base class?
like adding condition to use mcmc or vi? @janfb 

Resolving patch:
diff --git a/sbi/inference/snpe/snpe_a.py b/sbi/inference/snpe/snpe_a.py
--- a/sbi/inference/snpe/snpe_a.py
+++ b/sbi/inference/snpe/snpe_a.py
@@ -275,6 +275,7 @@ def build_posterior(
         self,
         density_estimator: Optional[TorchModule] = None,
         prior: Optional[Distribution] = None,
+        **kwargs,
     ) -> "DirectPosterior":
         r"""Build posterior from the neural density estimator.
 
@@ -300,11 +301,12 @@ def build_posterior(
         wrapped_density_estimator = self.correct_for_proposal(
             density_estimator=density_estimator
         )
-        self._posterior = DirectPosterior(
-            posterior_estimator=wrapped_density_estimator,  # type: ignore
+        self._posterior = super().build_posterior(
+            density_estimator=wrapped_density_estimator,
             prior=prior,
+            **kwargs,
         )
-        return deepcopy(self._posterior)
+        return deepcopy(self._posterior)  # type: ignore
 
     def _log_prob_proposal_posterior(
         self,


Test patch:
Repository: sbi-dev/sbi

Issue:
Default params of `check_sbc` produce torch UserWarning
**Describe the bug**

The default parameters of `check_sbc` produce a torch UserWarning with torch 2.2.1.
```
/home/steinb95/development/sbi-dev/repo/sbi/diagnostics/sbc.py:363: UserWarning: std(): degrees of freedom is <= 0. Correction should be strictly less than the reduction factor (input numel divided by output numel). (Triggered internally at ../aten/src/ATen/native/ReduceOps.cpp:1760.)
```
This is cased by https://github.com/sbi-dev/sbi/blob/1d25046a6a9834df28df7c142d2b072442921836/sbi/diagnostics/sbc.py#L363 as `sometensor.std(0)` is computed with only one value to apply to (due to `num_repetitions: int = 1` being the default).

**To Reproduce**

1. set up the `sbi` development environment
2. run [tutorial 13](https://github.com/sbi-dev/sbi/blob/main/tutorials/13_diagnostics_simulation_based_calibration.ipynb)
3. all cells invocing `check_sbc` will trigger this

**Expected behavior**
No warning




Discussion:

Resolving patch:
diff --git a/sbi/diagnostics/sbc.py b/sbi/diagnostics/sbc.py
--- a/sbi/diagnostics/sbc.py
+++ b/sbi/diagnostics/sbc.py
@@ -361,7 +361,8 @@ def check_uniformity_c2st(
     ])
 
     # Use variance over repetitions to estimate robustness of c2st.
-    if (c2st_scores.std(0) > 0.05).any():
+    c2st_std = c2st_scores.std(0, correction=0 if num_repetitions == 1 else 1)
+    if (c2st_std > 0.05).any():
         warnings.warn(
             f"""C2ST score variability is larger than {0.05}: std={c2st_scores.std(0)},
             result may be unreliable. Consider increasing the number of samples.


Test patch:
Repository: sbi-dev/sbi

Issue:
Improve tutorial on embedding networks
- make clear more early in the tutorial (and title) that this will be about embedding networks
- highlight more prominently pre-configured networks


Discussion:

Resolving patch:
diff --git a/sbi/neural_nets/density_estimators/mixed_density_estimator.py b/sbi/neural_nets/density_estimators/mixed_density_estimator.py
--- a/sbi/neural_nets/density_estimators/mixed_density_estimator.py
+++ b/sbi/neural_nets/density_estimators/mixed_density_estimator.py
@@ -164,8 +164,8 @@ def log_prob_iid(self, x: Tensor, context: Tensor) -> Tensor:
                 evaluated for the entire batch of iid x.
 
         Returns:
-            Tensor: log probs with shape (num_trials, num_parameters), i.e., the log
-                prob for each context for each trial.
+            log probs with shape (num_trials, num_parameters), i.e., the log prob for
+            each context for each trial.
         """
 
         context = atleast_2d(context)


Test patch:
Repository: sbi-dev/sbi

Issue:
PyMC samplers failing on simple Gaussian example.
**Describe the bug**
We are using a simple Gaussian posterior example to test the MCMC methods. The new PyMC samplers are failing for this example, producing c2st around 0.85. 

**To Reproduce**
Please add a minimal code example that reproduces the problem:
```bash
pytest tests/mcmc_test.py::test_c2st_pymc_sampler_on_Gaussian
```

**Expected behavior**
Test should pass. 

**Additional context**
See discussion here: #1124 



Discussion:

Resolving patch:
diff --git a/sbi/samplers/mcmc/pymc_wrapper.py b/sbi/samplers/mcmc/pymc_wrapper.py
--- a/sbi/samplers/mcmc/pymc_wrapper.py
+++ b/sbi/samplers/mcmc/pymc_wrapper.py
@@ -144,10 +144,9 @@ def __init__(
             potential_fn, track_gradients=track_gradients, device=device
         )
         with self._model:
-            params = pymc.Normal(
-                self.param_name, mu=initvals.mean(axis=0)
-            )  # dummy prior
-            pymc.Potential("likelihood", potential(params))  # type: ignore
+            pymc.DensityDist(
+                self.param_name, logp=potential, size=(initvals.shape[-1],)
+            )
 
     def run(self) -> np.ndarray:
         """Run MCMC with PyMC


Test patch:
Repository: frappe/frappe

Issue:
[fix] Validate email even if they are separated by new line



Discussion:

Resolving patch:
diff --git a/frappe/model/base_document.py b/frappe/model/base_document.py
--- a/frappe/model/base_document.py
+++ b/frappe/model/base_document.py
@@ -285,7 +285,7 @@ def _validate_update_after_submit(self):
 		current = frappe.db.get_value(self.doctype, self.name, "*", as_dict=True)
 		for key, value in current.iteritems():
 			df = self.meta.get_field(key)
-			if df and not df.allow_on_submit and self.get(key) != value:
+			if df and not df.allow_on_submit and (self.get(key) or value) and self.get(key) != value:
 				frappe.throw(_("Not allowed to change {0} after submission").format(df.label),
 					frappe.UpdateAfterSubmitError)
 
diff --git a/frappe/model/document.py b/frappe/model/document.py
--- a/frappe/model/document.py
+++ b/frappe/model/document.py
@@ -157,6 +157,8 @@ def save(self, ignore_permissions=None):
 		self.set_parent_in_children()
 		self.run_before_save_methods()
 		self._validate()
+		if self._action == "update_after_submit":
+			self.validate_update_after_submit()
 
 		# parent
 		if self.meta.issingle:
@@ -284,7 +286,6 @@ def check_docstatus_transition(self, docstatus):
 		elif docstatus==1:
 			if self.docstatus==1:
 				self._action = "update_after_submit"
-				self.validate_update_after_submit()
 				if not self.has_permission("submit"):
 					self.raise_no_permission_to("submit")
 			elif self.docstatus==2:
@@ -388,6 +389,7 @@ def run_before_save_methods(self):
 		elif self._action=="cancel":
 			self.run_method("before_cancel")
 		elif self._action=="update_after_submit":
+			self.run_method("validate")
 			self.run_method("before_update_after_submit")
 
 	def run_post_save_methods(self):


Test patch:
Repository: frappe/frappe

Issue:
Cannot use 'set only once' flag on doctype fields
I was trying to set some constant fields on a custom doctype using the 'set only once' flag. However, the framework is attempting to validate these fields against the non-yet-exists database record on create

The stack trace leads to [this function](https://github.com/frappe/frappe/commit/893588a0cef7e7adf8ecad83e90b8e082a50c734#diff-275b3c6438a6a531911b3900775bdabfR196).

I think this a one line fix; just amend the import condition to include create calls:
`
if frappe.flags.in_import or self.is_new():
` 



Discussion:

Resolving patch:
diff --git a/frappe/model/base_document.py b/frappe/model/base_document.py
--- a/frappe/model/base_document.py
+++ b/frappe/model/base_document.py
@@ -332,7 +332,7 @@ def _validate_selects(self):
 					value, comma_options))
 
 	def _validate_constants(self):
-		if frappe.flags.in_import:
+		if frappe.flags.in_import or self.is_new():
 			return
 
 		constants = [d.fieldname for d in self.meta.get("fields", {"set_only_once": 1})]


Test patch:
Repository: frappe/frappe

Issue:
Set Only Once for dates error on _validate_constants
_validate_constants always fails as the comparison is always returning false when the field type is Date

[https://github.com/frappe/frappe/blob/develop/frappe/model/base_document.py#L507](url)

**values.get(fieldname)**
Out[1]: 
datetime.date(2016, 11, 7)
**self.get(fieldname)**
Out[2]: 
u'2016-11-07'

In order to send PR, would you fix this here or in the "get" function?


Discussion:
Hi there, i'm facing the same issue; opened an issue some days ago.

refer to https://github.com/frappe/frappe/issues/2224

@paurosello fix it here, fixing it in `get` could cause bigger ripples else where.


Resolving patch:
diff --git a/frappe/model/base_document.py b/frappe/model/base_document.py
--- a/frappe/model/base_document.py
+++ b/frappe/model/base_document.py
@@ -504,7 +504,16 @@ def _validate_constants(self):
 			values = frappe.db.get_value(self.doctype, self.name, constants, as_dict=True)
 
 		for fieldname in constants:
-			if self.get(fieldname) != values.get(fieldname):
+			df = self.meta.get_field(fieldname)
+
+			# This conversion to string only when fieldtype is Date 
+			if df.fieldtype == 'Date':
+				value = str(values.get(fieldname))
+
+			else:
+				value  = values.get(fieldname)
+
+			if self.get(fieldname) != value:
 				frappe.throw(_("Value cannot be changed for {0}").format(self.meta.get_label(fieldname)),
 					frappe.CannotChangeConstantError)
 


Test patch:
Repository: frappe/frappe

Issue:
Move app installation to background
Long installs timeout the installation of the app and leads to broken installs.



Discussion:
I think we should kill the UI based app installer

Resolving patch:
diff --git a/frappe/desk/page/applications/applications.py b/frappe/desk/page/applications/applications.py
--- a/frappe/desk/page/applications/applications.py
+++ b/frappe/desk/page/applications/applications.py
@@ -11,6 +11,7 @@
 import json
 from frappe import _
 from distutils.spawn import find_executable
+from frappe.utils.background_jobs import enqueue
 
 @frappe.whitelist()
 def get_app_list():
@@ -73,6 +74,12 @@ def install_app(name):
 	if app_hooks.get('hide_in_installer'):
 		frappe.throw(_("You cannot install this app"))
 
+	enqueue('frappe.desk.page.applications.applications.start_install', name=name)
+
+	frappe.msgprint(_('Queued for install'))
+
+
+def start_install(name):
 	frappe.publish_realtime("install_app_progress", {"status": _("Installing App {0}").format(name)},
 		user=frappe.session.user)
 
@@ -81,6 +88,20 @@ def install_app(name):
 	frappe.publish_realtime("install_app_progress", {"status": _("{0} Installed").format(name)},
 		user=frappe.session.user)
 
+@frappe.whitelist()
+def remove_app(name):
+	"""Remove installed app"""
+	frappe.only_for("System Manager")
+
+	if name in frappe.get_installed_apps():
+		enqueue('frappe.desk.page.applications.applications.start_remove', name=name)
+
+	frappe.msgprint(_('Queued for backup and removing {0}').format(frappe.bold(name)))
+
+def start_remove(name):
+	frappe.installer.remove_app(app_name=name, yes=True)
+	frappe.publish_realtime('msgprint', _('App {0} removed').format(frappe.bold(name)))
+
 def get_app(name):
 	"""Get app using git clone and install it in bench environment"""
 	app_listing = get_app_listing()


Test patch:
Repository: frappe/frappe

Issue:
feat(list): show percent bar in list
<img width="986" alt="Screenshot 2019-04-30 at 10 33 56 AM" src="https://user-images.githubusercontent.com/140076/56941710-e91ce580-6b33-11e9-972e-21e793c0d7cb.png">



Discussion:

Resolving patch:
diff --git a/frappe/custom/doctype/customize_form/customize_form.py b/frappe/custom/doctype/customize_form/customize_form.py
--- a/frappe/custom/doctype/customize_form/customize_form.py
+++ b/frappe/custom/doctype/customize_form/customize_form.py
@@ -41,7 +41,6 @@
 	'reqd': 'Check',
 	'unique': 'Check',
 	'ignore_user_permissions': 'Check',
-	'in_filter': 'Check',
 	'in_list_view': 'Check',
 	'in_standard_filter': 'Check',
 	'hidden': 'Check',
diff --git a/frappe/model/__init__.py b/frappe/model/__init__.py
--- a/frappe/model/__init__.py
+++ b/frappe/model/__init__.py
@@ -14,7 +14,7 @@
 	'parent','parentfield','parenttype','idx','docstatus')
 integer_docfield_properties = ("reqd", "search_index", "in_list_view", "permlevel",
 	"hidden", "read_only", "ignore_user_permissions", "allow_on_submit", "report_hide",
-	"in_filter", "no_copy", "print_hide", "unique")
+	"no_copy", "print_hide", "unique")
 optional_fields = ("_user_tags", "_comments", "_assign", "_liked_by", "_seen")
 
 def rename(doctype, old, new, debug=False):
diff --git a/frappe/model/sync.py b/frappe/model/sync.py
--- a/frappe/model/sync.py
+++ b/frappe/model/sync.py
@@ -41,14 +41,14 @@ def sync_for(app_name, force=0, sync_everything = False, verbose=False, reset_pe
 	l = len(files)
 	if l:
 		for i, doc_path in enumerate(files):
-			import_file_by_path(doc_path, force=force, ignore_version=True, 
+			import_file_by_path(doc_path, force=force, ignore_version=True,
 				reset_permissions=reset_permissions)
 			#print module_name + ' | ' + doctype + ' | ' + name
 
 			frappe.db.commit()
 
 			# show progress bar
-			update_progress_bar("Updating {0}".format(app_name), i, l)
+			update_progress_bar("Updating DocTypes for {0}".format(app_name), i, l)
 
 		print ""
 


Test patch:
Repository: frappe/frappe

Issue:
More Informative Output For When MariaDB is Not Configured To Use Barracuda
Hello, this is more of an enhancement request from a new comer.

I had a hell of a time getting the Bench/Frappe/ERPNext stack to work on my computer. I believe I won't have wasted so much time if the message outputted to my console was more explicit. This is what was shown to me

`Please add this to MariaDB's my.cnf and restart MariaDB before proceeding`

`[mysqld]`
`innodb-file-format=barracuda`
`innodb-file-per-table=1`
`innodb-large-prefix=1`
`character-set-client-handshake = FALSE`
`character-set-server = utf8mb4`
`collation-server = utf8mb4_unicode_ci`

`[mysql]`
`default-character-set = utf8mb4`
`
If the software had told me that installation was not complete, it would have helped my debugging. It honestly looked like harmless information and I kept getting  ProgrammingError: (1146, .....) for as long as I hadn't discovered that MariaDb was not setup properly.

To fix this, I propose that we improve on the message by changing it to something like:
`Frappe installation has stalled because MariaDB needs to be configure to use Barracuda storage engine. `
`Please add the settings below to MariaDB's my.cnf and restart MariaDB before proceeding`

### System Information
Mariadb: (Ver 15.1 Distrib 10.1.22-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2)
Frappe version: 8.0.19
OS: Ubuntu Xenial


Discussion:

Resolving patch:
diff --git a/frappe/installer.py b/frappe/installer.py
--- a/frappe/installer.py
+++ b/frappe/installer.py
@@ -356,14 +356,25 @@ def check_if_ready_for_barracuda():
 		}.items():
 
 		if mariadb_variables.get(key) != value:
-			print "="*80
-			print "Please add this to MariaDB's my.cnf and restart MariaDB before proceeding"
-			print
-			print expected_config_for_barracuda
-			print "="*80
+			site = frappe.local.site
+			msg = ("Creation of your site - {x} failed because MariaDB is not properly {sep}"
+			       "configured to use the Barracuda storage engine. {sep}"
+			       "Please add the settings below to MariaDB's my.cnf, restart MariaDB then {sep}"
+			       "run `bench new-site site17.local` again.{sep2}"
+			       "").format(x=site, sep2="\n"*2, sep="\n")
+
+			print_db_config(msg, expected_config_for_barracuda)
 			sys.exit(1)
 			# raise Exception, "MariaDB needs to be configured!"
 
+
+def print_db_config(explanation, config_text):
+	print ("="*80)
+	print (explanation)
+	print (config_text)
+	print ("="*80)
+
+
 def extract_sql_gzip(sql_gz_path):
 	try:
 		subprocess.check_call(['gzip', '-d', '-v', '-f', sql_gz_path])


Test patch:
Repository: frappe/frappe

Issue:
[Minor]Hard coded 'site17.local' in pull #3138
`site17.local` is hard-coded in the `check_if_ready_for_barracuda`. `site17.local` should be replaced with `frappe.local.site`


Discussion:

Resolving patch:
diff --git a/frappe/installer.py b/frappe/installer.py
--- a/frappe/installer.py
+++ b/frappe/installer.py
@@ -360,7 +360,7 @@ def check_if_ready_for_barracuda():
 			msg = ("Creation of your site - {x} failed because MariaDB is not properly {sep}"
 			       "configured to use the Barracuda storage engine. {sep}"
 			       "Please add the settings below to MariaDB's my.cnf, restart MariaDB then {sep}"
-			       "run `bench new-site site17.local` again.{sep2}"
+			       "run `bench new-site {x}` again.{sep2}"
 			       "").format(x=site, sep2="\n"*2, sep="\n")
 
 			print_db_config(msg, expected_config_for_barracuda)


Test patch: