internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: magenta/magenta

Issue:
execfile() was removed from Python 3
https://github.com/tensorflow/magenta/blob/master/magenta/tools/pip/setup.py#L23


Discussion:
Looks like a relatively easy fix: https://stackoverflow.com/questions/436198/what-is-an-alternative-to-execfile-in-python-3

Resolving patch:
diff --git a/magenta/tools/pip/setup.py b/magenta/tools/pip/setup.py
--- a/magenta/tools/pip/setup.py
+++ b/magenta/tools/pip/setup.py
@@ -20,7 +20,8 @@
 # executing __init__.py, which will end up requiring a bunch of dependencies to
 # execute (e.g., tensorflow, pretty_midi, etc.).
 # Makes the __version__ variable available.
-execfile('magenta/version.py')
+with open('magenta/version.py') as in_file:
+    exec(in_file.read())
 
 
 REQUIRED_PACKAGES = [


Test patch:
Repository: magenta/magenta

Issue:
Type Error for some parameters when using the new Python 3 Magenta on Win7
 I run into the following  Type Error when using the new Python 3 Magenta on Win7

When running
performance_rnn_generate --config=multiconditioned_p
erformance_with_dynamics --bundle_file=multiconditioned_performance_with_dynamic
s.mag --output_dir=D:\Fmusic\Magenta\midi\ --num_outputs=1 --num_steps=1000 --no
tes_per_second='[3,5,10,5,3]' --pitch_class_histogram='[[2, 0, 1, 0, 1, 1, 0, 1,
 0, 1, 0, 1],[0, 0, 1, 2, 0, 1, 0, 0, 0, 0, 0, 0],[2, 0, 1, 0, 1, 1, 0, 1, 0, 1,
 0, 1]]'  --primer_melody="[60,62,64,65,67,69,71,72]"

both notes_per_second and pitch_class_histogram paramters cause that kin of error

 File "d:\programme\anaconda3\envs\magenta\lib\site-packages\google\protobuf\in
ternal\type_checkers.py", line 109, in CheckValue
    raise TypeError(message)
TypeError: "'[3,5,10,5,3]'" has type <class 'str'>, but expected one of: ((<class 'bytes'>,),)



Discussion:

Resolving patch:
diff --git a/magenta/models/performance_rnn/performance_rnn_generate.py b/magenta/models/performance_rnn/performance_rnn_generate.py
--- a/magenta/models/performance_rnn/performance_rnn_generate.py
+++ b/magenta/models/performance_rnn/performance_rnn_generate.py
@@ -221,9 +221,9 @@ def run_with_flags(generator):
         'histogram will be ignored: %s', FLAGS.pitch_class_histogram)
 
   if FLAGS.notes_per_second is not None:
-    generator_options.args['note_density'].byte_value = FLAGS.notes_per_second
+    generator_options.args['note_density'].string_value = FLAGS.notes_per_second
   if FLAGS.pitch_class_histogram is not None:
-    generator_options.args['pitch_histogram'].byte_value = (
+    generator_options.args['pitch_histogram'].string_value = (
         FLAGS.pitch_class_histogram)
 
   generator_options.args['temperature'].float_value = FLAGS.temperature
diff --git a/magenta/models/performance_rnn/performance_sequence_generator.py b/magenta/models/performance_rnn/performance_sequence_generator.py
--- a/magenta/models/performance_rnn/performance_sequence_generator.py
+++ b/magenta/models/performance_rnn/performance_sequence_generator.py
@@ -150,8 +150,8 @@ def _generate(self, input_sequence, generator_options):
 
     # Extract generation arguments from generator options.
     arg_types = {
-        'note_density': lambda arg: ast.literal_eval(arg.byte_value),
-        'pitch_histogram': lambda arg: ast.literal_eval(arg.byte_value),
+        'note_density': lambda arg: ast.literal_eval(arg.string_value),
+        'pitch_histogram': lambda arg: ast.literal_eval(arg.string_value),
         'temperature': lambda arg: arg.float_value,
         'beam_size': lambda arg: arg.int_value,
         'branch_factor': lambda arg: arg.int_value,


Test patch:
Repository: magenta/magenta

Issue:
Error in running Onsets and Frames Colab Notebook
Hi @cghawthorne
I am using your [Colab notebook](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/magenta/onsets_frames_transcription/onsets_frames_transcription.ipynb) to test your model but it stopped working a week ago.

Error on the inference section:
UnknownError: exceptions.AttributeError: 'module' object has no attribute 'logamplitude'
	 [[Node: wav_to_spec = PyFunc[Tin=[DT_STRING], Tout=[DT_FLOAT], token="pyfunc_1"](transform_wav_data_op)]]
	 [[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[?], [?,?,88], [?,?,88], [?], [?], [?,?,88], [?,?,229,1]], output_types=[DT_STRING, DT_FLOAT, DT_FLOAT, DT_INT32, DT_STRING, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](Iterator)]]

Thanks,
Bardia




Discussion:

Resolving patch:
diff --git a/magenta/version.py b/magenta/version.py
--- a/magenta/version.py
+++ b/magenta/version.py
@@ -17,4 +17,4 @@
 pulling in all the dependencies in __init__.py.
 """
 
-__version__ = '0.3.5'
+__version__ = '0.3.6'


Test patch:
Repository: magenta/magenta

Issue:
Pip installation fails due to librosa dependency
Hi,

I'm trying to install the magenta-gpu but when I did a pip install magenta-gpu:

**librosa 0.6.2 has requirement joblib>=0.12, but you'll have joblib 0.11 which is incompatible.**



Discussion:
It helps me for my windows env
This is addressed by #1251, but we still need to create a new version of the pip package.

Resolving patch:
diff --git a/magenta/version.py b/magenta/version.py
--- a/magenta/version.py
+++ b/magenta/version.py
@@ -17,4 +17,4 @@
 pulling in all the dependencies in __init__.py.
 """
 
-__version__ = '0.3.10'
+__version__ = '0.3.11'


Test patch:
Repository: magenta/magenta

Issue:
[Bug] Melody_rnn Create Dataset get_pipeline missing arg
`melody_rnn_create_dataset --config=basic_rnn --input=notesequences.tfrecord --output_dir=sequence_examples` fails with this error:
```
File "/Users/ericcacciavillani/anaconda3/envs/Magenta_Testing/lib/python3.6/site-packages/magenta/pipelines/note_sequence_pipelines.py", line 184, in transform
    for amount in self._transposition_range:
TypeError: 'float' object is not iterable
```
We're trying to train our own melody rnn and we get this error, also we don't see a mention of transposition range in the melody rnn readme?

Looks like in `melody_rnn_create_dataset.py` at line 52 its calling `melody_rnn_pipeline.get_pipeline(config, FLAGS.eval_ratio)`
but `get_pipeline` in `melody_rnn_pipeline.py` takes 3 args `config`, `transposition_range`, and `eval ratio` so it looks like transposition_range is being set as eval_ratio.

Working with @robindiddams on this.


Discussion:
@adarob is this something that happened in your recent refactor?

Resolving patch:
diff --git a/magenta/models/melody_rnn/melody_rnn_create_dataset.py b/magenta/models/melody_rnn/melody_rnn_create_dataset.py
--- a/magenta/models/melody_rnn/melody_rnn_create_dataset.py
+++ b/magenta/models/melody_rnn/melody_rnn_create_dataset.py
@@ -49,7 +49,7 @@ def main(unused_argv):
 
   config = melody_rnn_config_flags.config_from_flags()
   pipeline_instance = melody_rnn_pipeline.get_pipeline(
-      config, FLAGS.eval_ratio)
+      config, eval_ratio=FLAGS.eval_ratio)
 
   FLAGS.input = os.path.expanduser(FLAGS.input)
   FLAGS.output_dir = os.path.expanduser(FLAGS.output_dir)


Test patch:
Repository: magenta/magenta

Issue:
Python 3 support: MusicVAE evaluation
When launching the MusicVAE model in evaluation mode using:

`music_vae_train --config="cat-drums_2bar_small" --run_dir="logdir/run1" --mode="eval" --examples_path="sequence_examples/eval.tfrecord"`

I get the following error:

```
Traceback (most recent call last):
  File "/home/alex/miniconda3/envs/magenta/bin/music_vae_train", line 8, in <module>
    sys.exit(console_entry_point())
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/music_vae/music_vae_train.py", line 342, in console_entry_point
    tf.app.run(main)
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/music_vae/music_vae_train.py", line 338, in main
    run(configs.CONFIG_MAP)
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/music_vae/music_vae_train.py", line 333, in run
    master=FLAGS.master)
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/music_vae/music_vae_train.py", line 238, in evaluate
    **_get_input_tensors(dataset_fn().take(num_batches), config))
  File "/home/alex/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/music_vae/base_model.py", line 331, in eval
    for n, t in scalars_to_summarize.iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'
```

Which is a Python 3 compatibility issue. I already have the fix, I'm sending the PR soon.


Discussion:

Resolving patch:
diff --git a/magenta/models/music_vae/base_model.py b/magenta/models/music_vae/base_model.py
--- a/magenta/models/music_vae/base_model.py
+++ b/magenta/models/music_vae/base_model.py
@@ -328,16 +328,16 @@ def eval(self, input_sequence, output_sequence, sequence_length,
     metric_map, scalars_to_summarize = self._compute_model_loss(
         input_sequence, output_sequence, sequence_length, control_sequence)
 
-    for n, t in scalars_to_summarize.iteritems():
+    for n, t in scalars_to_summarize.items():
       metric_map[n] = tf.metrics.mean(t)
 
     metrics_to_values, metrics_to_updates = (
         contrib_metrics.aggregate_metric_map(metric_map))
 
-    for metric_name, metric_value in metrics_to_values.iteritems():
+    for metric_name, metric_value in metrics_to_values.items():
       tf.summary.scalar(metric_name, metric_value)
 
-    return metrics_to_updates.values()
+    return list(metrics_to_updates.values())
 
   def sample(self, n, max_length=None, z=None, c_input=None, **kwargs):
     """Sample with an optional conditional embedding `z`."""


Test patch:
Repository: magenta/magenta

Issue:
Error training latest version of MusicVAE
#### Steps to reproduce the issue

- Create new magenta environment as well as cloning magenta repo:
`curl https://raw.githubusercontent.com/tensorflow/magenta/master/magenta/tools/magenta-install.sh > /tmp/magenta-install.sh`
`bash /tmp/magenta-install.sh`
`git clone https://github.com/tensorflow/magenta.git`

- Training simple model:
`music_vae_train --config=cat-mel_2bar_small --run_dir=/tmp/cat-mel_2bar_small/ --mode=train --examples_path=data/notesequences.tfrecord`

#### Expected results:

- Training loss and global_step print on console.

#### Actual output:

- `line 123, in _get_input_tensors
    iterator = dataset.make_one_shot_iterator()
AttributeError: 'PrefetchDataset' object has no attribute 'make_one_shot_iterator'`


Discussion:
Can you try `pip install https://github.com/tensorflow/magenta/archive/master.zip`?
I'm having the same issue, just tried `pip install https://github.com/tensorflow/magenta/archive/master.zip` and got the same error.
Update: replacing `iterator = dataset.make_one_shot_iterator()` with `iterator = tf.data.make_one_shot_iterator(dataset)` seems to have fixed this. 

Running into this error now: #1622 
@willfenton do you want to make a PR or should I make that change? I'm looking into the second issue now, but i think we just have to add back disabling autograph until we can move to tf 2.x.
@gauravmishra the get_dataset test you added would ideally catch this autograph issue. I think you need to actually make a simple DataConverter instead of mocking the whole thing.

Resolving patch:
diff --git a/magenta/models/music_vae/music_vae_train.py b/magenta/models/music_vae/music_vae_train.py
--- a/magenta/models/music_vae/music_vae_train.py
+++ b/magenta/models/music_vae/music_vae_train.py
@@ -120,7 +120,7 @@ def _trial_summary(hparams, examples_path, output_dir):
 def _get_input_tensors(dataset, config):
   """Get input tensors from dataset."""
   batch_size = config.hparams.batch_size
-  iterator = dataset.make_one_shot_iterator()
+  iterator = tf.data.make_one_shot_iterator(dataset)
   (input_sequence, output_sequence, control_sequence,
    sequence_length) = iterator.get_next()
   input_sequence.set_shape(


Test patch:
Repository: magenta/magenta

Issue:
Retraining Onsets and Frames Drums model with E-GMD dataset
Hello,

I am trying to retrain OaF model with the E-GMD dataset for drums transcription. I first downloaded the E-GMD dataset which has its corresponding csv file and a directoy for each drummer and subdirectories with the sessions.

I am trying to do the first step following the code in ```onsets_frames_transcription_create_tfrecords``` which I found that it is:

```
onsets_frames_transcription_create_tfrecords \
  --csv=".../e-gmd-v1.0.0/e-gmd-v1.0.0.csv" \
  --output_directory=".../e-gmd-v1.0.0" \
  --num_shards="0" \
  --wav_dir=".../e-gmd-v1.0.0" \
  --midi_dir=".../e-gmd-v1.0.0" \
  --expected_splits="train,validation,test"
```

But i got the following error which I don't know where does it come from:

```
2020-08-05 17:23:45.289023: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-08-05 17:23:45.289348: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING:tensorflow:From c:\users\carlos\anaconda3\lib\site-packages\tensorflow\python\compat\v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
WARNING:tensorflow:From c:\users\carlos\anaconda3\lib\site-packages\tensorflow\python\compat\v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Traceback (most recent call last):
  File "c:\users\carlos\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\carlos\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Carlos\Anaconda3\Scripts\onsets_frames_transcription_create_tfrecords.exe\__main__.py", line 4, in <module>
ImportError: cannot import name 'console_entry_point'
```
I don't know if I have to change the paths of the wav and MIDI files in order to have the wav files in a directory and the MIDI files in other directory or the error comes from installation issues, versions, etc.

I am using Winows 10.


Discussion:

Resolving patch:
diff --git a/magenta/models/onsets_frames_transcription/onsets_frames_transcription_create_tfrecords.py b/magenta/models/onsets_frames_transcription/onsets_frames_transcription_create_tfrecords.py
--- a/magenta/models/onsets_frames_transcription/onsets_frames_transcription_create_tfrecords.py
+++ b/magenta/models/onsets_frames_transcription/onsets_frames_transcription_create_tfrecords.py
@@ -139,5 +139,10 @@ def main(argv):
           num_shards=FLAGS.num_shards)
 
 
-if __name__ == '__main__':
+def console_entry_point():
+  tf.disable_v2_behavior()
   app.run(main)
+
+
+if __name__ == '__main__':
+  console_entry_point()


Test patch:
Repository: talonhub/community

Issue:
feature request: `phrase hello slash world` -> `hello/world`
I'm not sure what the best way to go about this is, it might be simple.

some more examples/unit tests:

- `phrase hello slash world` -> `hello/world` (currently `hello /world`)
- `phrase hello slash snake name of file slash world` -> `hello/name_of_file/world` (currently `hello/name_of_file_/world`)

Similarly: `phrase file dot extension` -> `file.extension`, though this one might be complicated by the fact that I have a formatter called `dot` defined so that `dot hello world` -> `.hello_world` because I use python so much.


Discussion:

Resolving patch:
diff --git a/code/formatters.py b/code/formatters.py
--- a/code/formatters.py
+++ b/code/formatters.py
@@ -193,6 +193,22 @@ def format_text(m) -> str:
     "Formats the text and returns a string"
 
 
+class ImmuneString(object):
+    """Wrapper that makes a string immune from formatting."""
+
+    def __init__(self, string):
+        self.string = string
+
+
+@mod.capture
+def formatter_immune(m) -> ImmuneString:
+    """Text that can be interspersed into a formatter, e.g. characters.
+
+    It will be inserted directly, without being formatted.
+
+    """
+
+
 @mod.action_class
 class Actions:
     def formatted_text(phrase: Union[str, Phrase], formatters: str) -> str:
@@ -243,15 +259,45 @@ def formatters_reformat_selection(formatters: str) -> str:
         actions.insert(text)
         return text
 
+    def insert_many(strings: List[str]) -> None:
+        """Insert a list of strings, sequentially."""
+        for string in strings:
+            actions.insert(string)
+
 
 @ctx.capture(rule="{self.formatters}+")
 def formatters(m):
     return ",".join(m.formatters_list)
 
 
-@ctx.capture(rule="<self.formatters> <user.text>")
+@ctx.capture(
+    # Add anything else into this that you want to be able to speak during a
+    # formatter.
+    rule="(<user.symbol_key> | <user.letter> | numb <number>)"
+)
+def formatter_immune(m) -> ImmuneString:
+    if hasattr(m, "number"):
+        value = m.number
+    else:
+        value = m[0]
+    return ImmuneString(str(value))
+
+
+@ctx.capture(
+    # Note that if the user speaks something like "snake dot", it will
+    # insert "dot" - otherwise, they wouldn't be able to insert punctuation
+    # words directly.
+    rule="<self.formatters> <user.text> (<user.text> | <user.formatter_immune>)*"
+)
 def format_text(m):
-    return format_phrase(m.text, m.formatters)
+    out = ""
+    formatters = m[0]
+    for chunk in m[1:]:
+        if isinstance(chunk, ImmuneString):
+            out += chunk.string
+        else:
+            out += format_phrase(chunk, formatters)
+    return out
 
 
 ctx.lists["self.formatters"] = formatters_words.keys()


Test patch:
Repository: talonhub/community

Issue:
formatters should apply to selected text if they are spoken alone
for example `yeller` while some text is selected should pass the selected text through the formatter and replace the selected text with the formatted text.


Discussion:
from jsenecal:
```
Added this to formatters.py:
    def formatters_reformat_selection(formatters: str) -> str:
        """Reformats current selection"""
        return format_phrase(actions.edit.selected_text(), formatters)

Added this to  formatters.talon:
<user.formatters> that: insert(user.formatters_reformat_selection(user.formatters))
```



Resolving patch:
diff --git a/code/formatters.py b/code/formatters.py
--- a/code/formatters.py
+++ b/code/formatters.py
@@ -1,9 +1,11 @@
 from talon import Module, Context, actions, ui, imgui
 from talon.grammar import Phrase
 from typing import List, Union
+import re
 
 ctx = Context()
 key = actions.key
+edit = actions.edit
 
 words_to_keep_lowercase = "a,an,the,at,by,for,in,is,of,on,to,up,and,as,but,or,nor".split(
     ","
@@ -228,6 +230,19 @@ def formatters_reformat_last(formatters: str) -> str:
         global last_phrase
         return format_phrase(last_phrase, formatters)
 
+    def formatters_reformat_selection(formatters: str) -> str:
+        """Reformats the current selection."""
+        selected = edit.selected_text()
+        unformatted = re.sub(r"[^a-zA-Z0-9]+", " ", selected).lower()
+        # TODO: Separate out camelcase & studleycase vars
+
+        # Delete separately for compatibility with programs that don't overwrite
+        # selected text (e.g. Emacs)
+        edit.delete()
+        text = actions.self.formatted_text(unformatted, formatters)
+        actions.insert(text)
+        return text
+
 
 @ctx.capture(rule="{self.formatters}+")
 def formatters(m):


Test patch:
Repository: talonhub/community

Issue:
Vocabulary functions do not strip whitespace from CSV
I had the misfortune the last few days to believe that the vocabulary csv files were not operating correctly.

Apparently whitespace is not stripped around fields. So "test, test2" leads a row which is parsed, but not turned into a functioning command.


Discussion:
this might be the same as https://github.com/talonvoice/talon/issues/199, which was just fixed
> this might be the same as [talonvoice/talon#199](https://github.com/talonvoice/talon/issues/199), which was just fixed

I replicated this behaviour using that Talon release, it's still an issue.
I have a fix for this, I just need to make a PR for it, I should get to it sometime tomorrow.

Resolving patch:
diff --git a/code/user_settings.py b/code/user_settings.py
--- a/code/user_settings.py
+++ b/code/user_settings.py
@@ -51,14 +51,17 @@ def _load_csv_dict(
                 # Windows newlines are sometimes read as empty rows. :champagne:
                 continue
             if len(row) == 1:
-                mapping[row[0]] = row[0]
+                output = spoken_form = row[0]
             else:
-                mapping[row[1]] = row[0]
+                output, spoken_form = row[:2]
                 if len(row) > 2:
                     print(
                         f'"{file_name}": More than two values in row: {row}.'
                         + " Ignoring the extras."
                     )
+            # Leading/trailing whitespace in spoken form can prevent recognition.
+            spoken_form = spoken_form.strip()
+            mapping[spoken_form] = output
     return mapping
 
 


Test patch:
Repository: talonhub/community

Issue:
the command key should probably be made Mac specific



Discussion:

Resolving patch:
diff --git a/code/keys.py b/code/keys.py
--- a/code/keys.py
+++ b/code/keys.py
@@ -111,15 +111,17 @@ def letters(m) -> str:
 
 
 ctx = Context()
-ctx.lists["self.modifier_key"] = {
+modifier_keys = {
     # If you find 'alt' is often misrecognized, try using 'alter'.
     "alt": "alt",  #'alter': 'alt',
-    "command": "cmd",
     "control": "ctrl",  #'troll':   'ctrl',
-    "option": "alt",
     "shift": "shift",  #'sky':     'shift',
     "super": "super",
 }
+if app.platform  == "mac":
+    modifier_keys["command"] = "cmd"
+    modifier_keys["option"] = "alt"
+ctx.lists["self.modifier_key"] = modifier_keys
 alphabet = dict(zip(default_alphabet, letters_string))
 ctx.lists["self.letter"] = alphabet
 


Test patch: