internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: archlinux/archinstall

Issue:
Don't check if file path ends with .json
When trying to use a path to a file that does not end with `.json`, the script interprets it as a json string, rather then a file path.

Relevant code: https://github.com/archlinux/archinstall/blob/7f0e031c66e3f5adb1cf72542b1fde7e86aa5116/examples/guided.py#L54

Quoted from discord:
> We could probably omit the check for .endswith('.json') and assume that if pathlib.Path().exists() is true, it is a file and that it should be tried as a JSON file

https://discord.com/channels/726808923662712894/726809631342592081/930105481794555954


Discussion:

Resolving patch:
diff --git a/examples/guided.py b/examples/guided.py
--- a/examples/guided.py
+++ b/examples/guided.py
@@ -51,7 +51,7 @@ def load_config():
 	if archinstall.arguments.get('servers', None) is not None:
 		archinstall.storage['_selected_servers'] = archinstall.arguments.get('servers', None)
 	if archinstall.arguments.get('disk_layouts', None) is not None:
-		if (dl_path := pathlib.Path(archinstall.arguments['disk_layouts'])).exists() and str(dl_path).endswith('.json'):
+		if (dl_path := pathlib.Path(archinstall.arguments['disk_layouts'])).exists():
 			try:
 				with open(dl_path) as fh:
 					archinstall.storage['disk_layouts'] = json.load(fh)


Test patch:
Repository: archlinux/archinstall

Issue:
archinstall formats as vfat but then doesn't understand it
I know that vfat and fat32 are interchangeable because I can manually install Arch doing either mkfs.vfat /dev/sda1 or mkfs.fat -F32 /dev/sda1.

archinstall does not recognize vfat as a valid file system if you choose option 8 to select the file system for the partition.  In that case you have to use fat32.  Once you do that, and install your system, all is good.

The next time you boot the USB key and run archinstall it shows that same partition as vfat and if you leave it alone, the install will fail.
This is discussed on the forum at https://bbs.archlinux.org/viewtopic.php?id=272794


Discussion:
We can probably use this helper function to map it for us: https://github.com/archlinux/archinstall/blob/7f9b7991e902489efb3501a98a7d6998ca15a0a5/archinstall/lib/disk/partition.py#L423-L428

or the safest bet for the `v2.3.1` release is to just double down on the if case here: https://github.com/archlinux/archinstall/blob/7f9b7991e902489efb3501a98a7d6998ca15a0a5/archinstall/lib/disk/partition.py#L286

Resolving patch:
diff --git a/archinstall/lib/disk/partition.py b/archinstall/lib/disk/partition.py
--- a/archinstall/lib/disk/partition.py
+++ b/archinstall/lib/disk/partition.py
@@ -268,6 +268,9 @@ def format(self, filesystem=None, path=None, log_formatting=True, options=[]):
 		if path is None:
 			path = self.path
 
+		# This converts from fat32 -> vfat to unify filesystem names
+		filesystem = get_mount_fs_type(filesystem)
+
 		# To avoid "unable to open /dev/x: No such file or directory"
 		start_wait = time.time()
 		while pathlib.Path(path).exists() is False and time.time() - start_wait < 10:
@@ -283,7 +286,7 @@ def format(self, filesystem=None, path=None, log_formatting=True, options=[]):
 				raise DiskError(f'Could not format {path} with {filesystem} because: {mkfs}')
 			self.filesystem = filesystem
 
-		elif filesystem == 'fat32':
+		elif filesystem == 'vfat':
 			options = ['-F32'] + options
 
 			mkfs = SysCommand(f"/usr/bin/mkfs.vfat {' '.join(options)} {path}").decode('UTF-8')
@@ -319,7 +322,7 @@ def format(self, filesystem=None, path=None, log_formatting=True, options=[]):
 				raise DiskError(f"Could not format {path} with {filesystem} because: {handle.decode('UTF-8')}")
 			self.filesystem = filesystem
 
-		elif filesystem == 'ntfs':
+		elif filesystem == 'ntfs3':
 			options = ['-f'] + options
 
 			if (handle := SysCommand(f"/usr/bin/mkfs.ntfs -Q {' '.join(options)} {path}")).exit_code != 0:


Test patch:
Repository: archlinux/archinstall

Issue:
desktop profiles, replace pipewire-media-session with wireplumber, wireplumber seembs to be the newest replacement media-session replaceing pipewire-media-session
hi,
would it be possible to replace pipewire-media-session with wireplumber?
or, to make things flexable,
add a question in asking,
"would you like to use wireplumber or pipewire-media-session?
p.s
i'm using wireplumber currently.
wireplumber will remove pipewire-media-session and the systemd units.
a total replacement.
thank you for considdering my request!
Majid Hussain


Discussion:
hi,
found this about wireplumber
Two session managers are now available. The one pulled in by default pipewire-media-session is primitive, and is best when using PipeWire just for its basic functionality like screensharing. When using PipeWire as your system’s sound server, the maintainer recommends installing the more advanced WirePlumber instead.
We probably should switch pipewire-media-session with wireplumber. It looks like the wireplumber package is the new official solution. https://archlinux.org/packages/extra/x86_64/wireplumber/

Resolving patch:
diff --git a/profiles/applications/pipewire.py b/profiles/applications/pipewire.py
--- a/profiles/applications/pipewire.py
+++ b/profiles/applications/pipewire.py
@@ -3,7 +3,7 @@
 
 # Define the package list in order for lib to source
 # which packages will be installed by this profile
-__packages__ = ["pipewire", "pipewire-alsa", "pipewire-jack", "pipewire-media-session", "pipewire-pulse", "gst-plugin-pipewire", "libpulse"]
+__packages__ = ["pipewire", "pipewire-alsa", "pipewire-jack", "pipewire-pulse", "gst-plugin-pipewire", "libpulse", "wireplumber"]
 
 archinstall.log('Installing pipewire', level=logging.INFO)
 archinstall.storage['installation_session'].add_additional_packages(__packages__)
@@ -11,4 +11,4 @@
 @archinstall.plugin
 def on_user_created(installation :archinstall.Installer, user :str):
 	archinstall.log(f"Enabling pipewire-pulse for {user}", level=logging.INFO)
-	installation.chroot('systemctl enable --user pipewire-pulse.service', run_as=user)
\ No newline at end of file
+	installation.chroot('systemctl enable --user pipewire-pulse.service', run_as=user)


Test patch:
Repository: archlinux/archinstall

Issue:
For 2.4.0, if it's a BIOS system and only GRUB is available, default the selection to grub-install



Discussion:
I agree, and overall we should populate the empty selections with default values so that there is no “x missing” on the install button. Let’s default to systemd-boot when possible and default to host name “archinstall” for instance. 
> I agree, and overall we should populate the empty selections with default values so that there is no “x missing” on the install button. Let’s default to systemd-boot when possible and default to host name “archinstall” for instance.

Maybe 'archlinux' as the hostname? :) It's the target system we're talking about here, not the installer
Fair enough heh, saw an opportunity for humble brag ^^ But we are installing Arch and it's out parent project so it makes sense. `archlinux` would be good :)

Resolving patch:
diff --git a/archinstall/lib/menu/selection_menu.py b/archinstall/lib/menu/selection_menu.py
--- a/archinstall/lib/menu/selection_menu.py
+++ b/archinstall/lib/menu/selection_menu.py
@@ -6,6 +6,7 @@
 
 from .menu import Menu
 from ..general import SysCommand, secret
+from ..hardware import has_uefi
 from ..storage import storage
 from ..output import log
 from ..profiles import is_desktop_profile
@@ -454,7 +455,8 @@ def _setup_selection_menu_options(self):
 		self._menu_options['bootloader'] = \
 			Selector(
 				_('Select bootloader'),
-				lambda: ask_for_bootloader(storage['arguments'].get('advanced', False)),)
+				lambda: ask_for_bootloader(storage['arguments'].get('advanced', False)),
+				default="systemd-bootctl" if has_uefi() else "grub-install")
 		self._menu_options['hostname'] = \
 			Selector(_('Specify hostname'), lambda: ask_hostname())
 		self._menu_options['!root-password'] = \
diff --git a/examples/guided.py b/examples/guided.py
--- a/examples/guided.py
+++ b/examples/guided.py
@@ -55,7 +55,7 @@ def ask_user_questions():
 	# Get disk encryption password (or skip if blank)
 	global_menu.enable('!encryption-password')
 
-	# Ask which boot-loader to use (will only ask if we're in BIOS (non-efi) mode)
+	# Ask which boot-loader to use (will only ask if we're in UEFI mode, otherwise will default to GRUB)
 	global_menu.enable('bootloader')
 
 	global_menu.enable('swap')


Test patch:
Repository: archlinux/archinstall

Issue:
Errors on boot, fstab corrupted due to missing genfstab
I don't really understand how I was able to boot and login at all, but I got to the login prompt after hitting enter a few times when presented with errors from trying to read `/etc/fstab`, and some error about unable to drop into a console as I went with the default of no root user.

Once I was logged in, I found `/etc/fstab` just contained a python traceback - `genfstab` not found. I installed `extra/arch-install-scripts` and `genfstab`bed it myself, and now seems fine.

Culprit here:
https://github.com/archlinux/archinstall/blob/ba725517fd290a60cd4e1ea570dbbf94a47ede05/archinstall/lib/installer.py#L187

Exit code is ignored and stderr written to the file.

(And also of course it needs to be installed so that it doesn't error anyway!)


Discussion:
You don't happen to have the traceback? : )

The code can be improved without it and I'll get on it, but it would be nice to see what caused the issue to begin with.
Sorry, I `tee`d `genfstab`s output into and indeed kicked myself for losing it.

You could recreate roughly the same just by mangling the path to `genfstab` on that line (or just uninstalling it) I suppose.
Hehe, easily happen! I'll put this on my todo-list and fix it soon. Might not make it to the release planned for this weekend. But at least the one to follow.

Resolving patch:
diff --git a/archinstall/lib/installer.py b/archinstall/lib/installer.py
--- a/archinstall/lib/installer.py
+++ b/archinstall/lib/installer.py
@@ -380,11 +380,14 @@ def set_mirrors(self, mirrors :Mapping[str, Iterator[str]]) -> None:
 	def genfstab(self, flags :str = '-pU') -> bool:
 		self.log(f"Updating {self.target}/etc/fstab", level=logging.INFO)
 
+		if not (fstab := SysCommand(f'/usr/bin/genfstab {flags} {self.target}')).exit_code == 0:
+			raise RequirementError(f'Could not generate fstab, strapping in packages most likely failed (disk out of space?)\n Error: {fstab}')
+
 		with open(f"{self.target}/etc/fstab", 'a') as fstab_fh:
-			fstab_fh.write((fstab := SysCommand(f'/usr/bin/genfstab {flags} {self.target}')).decode())
+			fstab_fh.write(fstab.decode())
 
 		if not os.path.isfile(f'{self.target}/etc/fstab'):
-			raise RequirementError(f'Could not generate fstab, strapping in packages most likely failed (disk out of space?)\n{fstab}')
+			raise RequirementError(f'Could not generate fstab, strapping in packages most likely failed (disk out of space?)\n Error: {fstab}')
 
 		for plugin in plugins.values():
 			if hasattr(plugin, 'on_genfstab'):
diff --git a/archinstall/lib/mirrors.py b/archinstall/lib/mirrors.py
--- a/archinstall/lib/mirrors.py
+++ b/archinstall/lib/mirrors.py
@@ -150,7 +150,7 @@ def list_mirrors(sort_order :List[str] = ["https", "http"]) -> Dict[str, Any]:
 	try:
 		response = urllib.request.urlopen(url)
 	except urllib.error.URLError as err:
-		log(f'Could not fetch an active mirror-list: {err}', level=logging.WARNING, fg="yellow")
+		log(f'Could not fetch an active mirror-list: {err}', level=logging.WARNING, fg="orange")
 		return regions
 
 	mirrorlist = response.read()


Test patch:
Repository: archlinux/archinstall

Issue:
Crash when password is or has a quotation mark.
Hi,
Hopefully this bug is easy to reproduce.   You just have to set a quotation mark (") as password.
 


Discussion:
[install.log](https://github.com/archlinux/archinstall/files/6689205/install.log)

I'll have a look at it, probably an issue in the way we escape strings.
Yes exactly, i have had this error a couple of times when using an uneven amount of quotation marks.
Is this still looked at?
Forgot to tag it for a release, Ill get on it!
Thanks!

    Just fyi:
    When using a password like: 1234!"asdf"
    the password comes out as 1234!
    and when using a password with uneven amount for example 1234!"asdf. The script crashes!

Resolving patch:
diff --git a/archinstall/lib/installer.py b/archinstall/lib/installer.py
--- a/archinstall/lib/installer.py
+++ b/archinstall/lib/installer.py
@@ -1028,7 +1028,12 @@ def user_set_pw(self, user :str, password :str) -> bool:
 			# This means the root account isn't locked/disabled with * in /etc/passwd
 			self.helper_flags['user'] = True
 
-		return SysCommand(f"/usr/bin/arch-chroot {self.target} sh -c \"echo '{user}:{password}' | chpasswd\"").exit_code == 0
+		combo = f'{user}:{password}'
+		echo = shlex.join(['echo', combo])
+		sh = shlex.join(['sh', '-c', echo])
+		
+		result = SysCommand(f"/usr/bin/arch-chroot {self.target} " + sh[:-1] + " | chpasswd'")
+		return result.exit_code == 0
 
 	def user_set_shell(self, user :str, shell :str) -> bool:
 		self.log(f'Setting shell for {user} to {shell}', level=logging.INFO)


Test patch:
Repository: archlinux/archinstall

Issue:
`--removable` in `grub-install` fails on some hardware
The full thread can be found here: https://discord.com/channels/726808923662712894/726809631342592081/937766587044364321
But here are some exerpts:
 * Log: https://0x0.st/oHEp.log

We'll need to detect the need for `--removable` or retry if it fails, here's the relevant code: https://github.com/archlinux/archinstall/blob/1515fb9fdeeb807b1301bc803d625dfebca8aaca/archinstall/lib/installer.py#L770


Discussion:
I feel a good idea would be to enable the `--debug` flag just so people can get an idea of what went wrong with grub in the future, but its also just nice to see it compile. Makes me calm at least. This is just my opinion, won't fix the bug aforementioned above.
Problem with --debug is we would have to flush it to stream, might just add to the complexity of the command overall. 
We can in a pretty simplistic way add `peak_output=True` to any command we run, which will render it to stdout. We can do that for the grub commands.

I’m in the process of refactoring `SysCommand` so that the forking doesn’t break. Currently we’re getting quite lucky that it doesn’t run commands twice and stuff. So no one will know, but that needs to be addressed before people know :)
Awesome, I look forward to that.

Resolving patch:
diff --git a/archinstall/lib/installer.py b/archinstall/lib/installer.py
--- a/archinstall/lib/installer.py
+++ b/archinstall/lib/installer.py
@@ -832,12 +832,15 @@ def add_grub_bootloader(self, boot_partition :Partition, root_partition :Partiti
 		if has_uefi():
 			self.pacstrap('efibootmgr') # TODO: Do we need? Yes, but remove from minimal_installation() instead?
 			try:
-				SysCommand(f'/usr/bin/arch-chroot {self.target} grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB --removable')
-			except SysCallError as error:
-				raise DiskError(f"Could not install GRUB to {self.target}/boot: {error}")
+				SysCommand(f'/usr/bin/arch-chroot {self.target} grub-install --debug --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB --removable', peak_output=True)
+			except SysCallError:
+				try:
+					SysCommand(f'/usr/bin/arch-chroot {self.target} grub-install --debug --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB --removable', peak_output=True)
+				except SysCallError as error:
+					raise DiskError(f"Could not install GRUB to {self.target}/boot: {error}")
 		else:
 			try:
-				SysCommand(f'/usr/bin/arch-chroot {self.target} grub-install --target=i386-pc --recheck {boot_partition.parent}')
+				SysCommand(f'/usr/bin/arch-chroot {self.target} grub-install --debug --target=i386-pc --recheck {boot_partition.parent}', peak_output=True)
 			except SysCallError as error:
 				raise DiskError(f"Could not install GRUB to {boot_partition.path}: {error}")
 


Test patch:
Repository: archlinux/archinstall

Issue:
Installer.zram_enabled is not set when swap is disabled
When trying to install with `"swap": False`, the installer fails prematurely when trying to install systemd-boot as it tries to access `self.zram_enabled`. This attribute is only set in `setup_swap()`, which does not get executed with swap disabled resulting in an exception.

logs: https://paste.dymstro.nl/?9fd3ee57a10a7725#5QguRT8KXp6jGQombKw7d2aDQmnjksrGC1TxsfFUFEkq


Discussion:
Thank you! I'll throw in a quick fix tomorrow!

Resolving patch:
diff --git a/archinstall/lib/installer.py b/archinstall/lib/installer.py
--- a/archinstall/lib/installer.py
+++ b/archinstall/lib/installer.py
@@ -129,6 +129,8 @@ def __init__(self, target :str, *, base_packages :Optional[List[str]] = None, ke
 		self.HOOKS = ["base", "udev", "autodetect", "keyboard", "keymap", "modconf", "block", "filesystems", "fsck"]
 		self.KERNEL_PARAMS = []
 
+		self._zram_enabled = False
+
 	def log(self, *args :str, level :int = logging.DEBUG, **kwargs :str):
 		"""
 		installer.log() wraps output.log() mainly to set a default log-level for this install session.
@@ -713,7 +715,7 @@ def setup_swap(self, kind :str = 'zram') -> bool:
 
 			self.enable_service('systemd-zram-setup@zram0.service')
 
-			self.zram_enabled = True
+			self._zram_enabled = True
 
 			return True
 		else:
@@ -793,7 +795,7 @@ def add_systemd_bootloader(self, boot_partition :Partition, root_partition :Part
 				# Zswap should be disabled when using zram.
 				#
 				# https://github.com/archlinux/archinstall/issues/881
-				if self.zram_enabled:
+				if self._zram_enabled:
 					options_entry = "zswap.enabled=0 " + options_entry
 
 				if real_device := self.detect_encryption(root_partition):
@@ -1038,7 +1040,7 @@ def user_set_pw(self, user :str, password :str) -> bool:
 		combo = f'{user}:{password}'
 		echo = shlex.join(['echo', combo])
 		sh = shlex.join(['sh', '-c', echo])
-		
+
 		result = SysCommand(f"/usr/bin/arch-chroot {self.target} " + sh[:-1] + " | chpasswd'")
 		return result.exit_code == 0
 


Test patch:
Repository: archlinux/archinstall

Issue:
Cyrillic problems (?)
I decided to test new translation and got this 
![Снимок экрана от 2022-05-06 17-47-09](https://user-images.githubusercontent.com/99257010/167248513-44b773ea-2f33-41b5-aa17-fc09a118358c.png)



Discussion:
Humm, possibly a missing font on the official ISO.
Do you know which font you typically use in these regions?
Just want to mention that you should choose font which has support for other languages which use Cyrillic outside Russia. 
Open Sans, Noto Sans, Roboto, Ubuntu. That's the first thing that came to mind.
I assume that some of these can handle Cyrillic as well. And most of them should handle other languages as well.
I was under the impression that only these would support Cyrillic: https://wiki.archlinux.org/title/fonts#Cyrillic
Noto Fonts are developed by Google and are designed for a large number of languages, including Cyrillic https://fonts.google.com/noto
I wonder if this is related to the rendering issue that is seen in french for certain characters 
Maybe the encoding isn't right when the .mo files are generated
> Maybe the encoding isn't right when the .mo files are generated

Might be, since some of these characters probably won't work in UTF-8?
> Maybe the encoding isn't right when the .mo files are generated

UTF-8 encoding.  The .mo file was made with locales_generator.sh and Poedit 3.0.1 
@Alexmelman88 I gave this a try and it seemed to work for me on a real machine but fails with the ISO inside a VM, so it might actually be a font issue as @Torxed pointed out. 
Would you be able to confirm if you can type cyrillic characters outside archinstall, basically just in the console of the terminal?
On real machine after these commands everything worked
![20220508_133548](https://user-images.githubusercontent.com/99257010/167292937-39ffc85d-820d-44f7-9607-fcf18aaa6956.jpg)

The setfont parameter probably helped
![20220508_134417](https://user-images.githubusercontent.com/99257010/167293060-0f7876ab-113c-43ee-8a9d-2df4c4430c5b.jpg)

Not all translation strings are listed in base.pot. For example "Press "/" to search", "True", "None".
Interesting, we might be able to load the font dynamically in the installer.

Those strings were probably missed, they have to be marked with a special wrapper to show up in the translation file, something like `str(_('something'))`

Resolving patch:
diff --git a/archinstall/__init__.py b/archinstall/__init__.py
--- a/archinstall/__init__.py
+++ b/archinstall/__init__.py
@@ -55,6 +55,10 @@
 DeferredTranslation.install()
 
 
+def set_unicode_font():
+	SysCommand('setfont UniCyr_8x16')
+
+
 def define_arguments():
 	"""
 	Define which explicit arguments do we allow.
@@ -243,6 +247,9 @@ def post_process_arguments(arguments):
 	load_config()
 
 
+# to ensure that cyrillic characters work in the installer
+set_unicode_font()
+
 define_arguments()
 arguments = get_arguments()
 post_process_arguments(arguments)


Test patch:
Repository: archlinux/archinstall

Issue:
failing on copy of boot manager
Available partitions: {'/mnt/archinstall': Partition(path=/dev/nvme0n1p6, size=888.9, PARTUUID=1a21560d-c9e9-4d5d-be40-01d66948963e, fs=ext4, mounted=/mnt/archinstall), '/mnt/archinstall/boot': Partition(path=/dev/nvme0n1p4, size=0.3, PARTUUID=50f513f3-7bfc-419e-968d-31953d79a1d3, fs=vfat, mounted=/mnt/archinstall/boot)}
Adding bootloader systemd-bootctl to Partition(path=/dev/nvme0n1p4, size=0.3, PARTUUID=50f513f3-7bfc-419e-968d-31953d79a1d3, fs=vfat, mounted=/mnt/archinstall/boot)
Installing packages: ('efibootmgr',)
['/usr/bin/arch-chroot', '/mnt/archinstall', 'bootctl', '--path=/boot', 'install'] exited with abnormal exit code [256]: b'Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/systemd-bootx64.efi".\r\nCopied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/BOOTX64.EFI".\r\nRandom seed file /boot/loader/random-seed successfully written (512 bytes).\r\nFailed to write \'LoaderSystemToken\' EFI variable: Read-only file system\r\n'


Discussion:
This is a triple boot installation on a MBP using the archinstall-mbp profile to do guided install.  Looks like a lot of install is actually happening as the / drive and the boot/efi are quite well populated.



Resolving patch:
diff --git a/archinstall/lib/installer.py b/archinstall/lib/installer.py
--- a/archinstall/lib/installer.py
+++ b/archinstall/lib/installer.py
@@ -775,7 +775,9 @@ def add_systemd_bootloader(self, boot_partition :Partition, root_partition :Part
 		# And in which case we should do some clean up.
 
 		# Install the boot loader
-		if SysCommand(f'/usr/bin/arch-chroot {self.target} bootctl --path=/boot install').exit_code != 0:
+		try:
+			SysCommand(f'/usr/bin/arch-chroot {self.target} bootctl --path=/boot install')
+		except SysCallError:
 			# Fallback, try creating the boot loader without touching the EFI variables
 			SysCommand(f'/usr/bin/arch-chroot {self.target} bootctl --no-variables --path=/boot install')
 


Test patch: