internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: archlinux/archinstall

Issue:
Issue installing package groups (kde-applications for instance)
As mentioned in #61, support for package groups doesn't work.
The idea here is that it should be supported, we simply never verified that the [archinstall.find_package()](https://github.com/Torxed/archinstall/blob/master/archinstall/lib/packages.py#L7-L17) function can verify those, and apparently it can't. So we have to use another API endpoint or multiple to support this.

*The backplane supports it already, as the packages are sent as a unfiltered string to `pacman -S` more or less.*


Discussion:

Resolving patch:
diff --git a/archinstall/lib/packages.py b/archinstall/lib/packages.py
--- a/archinstall/lib/packages.py
+++ b/archinstall/lib/packages.py
@@ -3,6 +3,23 @@
 from .exceptions import *
 
 BASE_URL = 'https://www.archlinux.org/packages/search/json/?name={package}'
+BASE_GROUP_URL = 'https://www.archlinux.org/groups/x86_64/{group}/'
+
+def find_group(name):
+	ssl_context = ssl.create_default_context()
+	ssl_context.check_hostname = False
+	ssl_context.verify_mode = ssl.CERT_NONE
+	try:
+		response = urllib.request.urlopen(BASE_GROUP_URL.format(group=name), context=ssl_context)
+	except urllib.error.HTTPError as err:
+		if err.code == 404:
+			return False
+		else:
+			raise err
+	
+	# Just to be sure some code didn't slip through the exception
+	if response.code == 200:
+		return True
 
 def find_package(name):
 	"""
@@ -34,7 +51,7 @@ def validate_package_list(packages :list):
 	"""
 	invalid_packages = []
 	for package in packages:
-		if not find_package(package)['results']:
+		if not find_package(package)['results'] and not find_group(package):
 			invalid_packages.append(package)
 	
 	if invalid_packages:


Test patch:
Repository: archlinux/archinstall

Issue:
Improve selecting from large lists
The search feature by entering `?<enter>` is a bit unclear.
And the scrollback history via `Shift+PageUp` is not always available *(in most tty's)*.

It would there for be nice to break up larger lists into columns over the width of the terminal, as well as make the input text more descriptive.

![unknown](https://user-images.githubusercontent.com/861439/105330634-d1707500-5bd2-11eb-9503-653c2f247026.png)



Discussion:

Resolving patch:
diff --git a/archinstall/lib/user_interaction.py b/archinstall/lib/user_interaction.py
--- a/archinstall/lib/user_interaction.py
+++ b/archinstall/lib/user_interaction.py
@@ -27,6 +27,22 @@ def get_password(prompt="Enter a password: "):
 		return passwd
 	return None
 
+def print_large_list(options, padding=5, margin_bottom=0, separator=': '):
+	highest_index_number_length = len(str(len(options)))
+	longest_line = highest_index_number_length + len(separator) + get_longest_option(options) + padding
+	max_num_of_columns = get_terminal_width() // longest_line
+	max_options_in_cells = max_num_of_columns * (get_terminal_height()-margin_bottom)
+
+	if (len(options) > max_options_in_cells):
+		for index, option in enumerate(options):
+			print(f"{index}: {option}")
+	else:
+		for row in range(0, (get_terminal_height()-margin_bottom)):
+			for column in range(row, len(options), (get_terminal_height()-margin_bottom)):
+				spaces = " "*(longest_line - len(options[column]))
+				print(f"{str(column): >{highest_index_number_length}}{separator}{options[column]}", end = spaces)
+			print()
+
 def ask_for_superuser_account(prompt='Create a required super-user with sudo privileges: ', forced=False):
 	while 1:
 		new_user = input(prompt).strip(' ')
@@ -279,8 +295,7 @@ def select_mirror_regions(mirrors, show_top_mirrors=True):
 	selected_mirrors = {}
 
 	if len(regions) >= 1:
-		for index, region in enumerate(regions):
-			print(f"{index}: {region}")
+		print_large_list(regions, margin_bottom=4)
 
 		print(' -- You can enter ? or help to search for more regions --')
 		print(' -- You can skip this step by leaving the option blank --')


Test patch:
Repository: archlinux/archinstall

Issue:
Missing range check when asked to pick an option
The dialog asking me to pick an option by typing its number isn't checking if the choice actually exists, causing an IndexError.

Tested with 2.1.3-1


Discussion:
> The dialog asking me to pick an option by typing its number 

You don't happen to remember which dialogue? Because most of them should have a range check.
I can go over them, they're not to many but if you recall which one - that would be very helpful :)

Resolving patch:
diff --git a/archinstall/lib/user_interaction.py b/archinstall/lib/user_interaction.py
--- a/archinstall/lib/user_interaction.py
+++ b/archinstall/lib/user_interaction.py
@@ -170,7 +170,10 @@ def generic_select(options, input_text="Select one of the above by index or abso
 	if len(selected_option.strip()) <= 0:
 		return None
 	elif selected_option.isdigit():
-		selected_option = options[int(selected_option)]
+		selected_option = int(selected_option)
+		if selected_option >= len(options):
+			raise RequirementError(f'Selected option "{selected_option}" is out of range')
+		selected_option = options[selected_option]
 	elif selected_option in options:
 		pass # We gave a correct absolute value
 	else:
@@ -195,7 +198,10 @@ def select_disk(dict_o_disks):
 			print(f"{index}: {drive} ({dict_o_disks[drive]['size'], dict_o_disks[drive].device, dict_o_disks[drive]['label']})")
 		drive = input('Select one of the above disks (by number or full path): ')
 		if drive.isdigit():
-			drive = dict_o_disks[drives[int(drive)]]
+			drive = int(drive)
+			if drive >= len(drives):
+				raise DiskError(f'Selected option "{drive}" is out of range')
+			drive = dict_o_disks[drives[drive]]
 		elif drive in dict_o_disks:
 			drive = dict_o_disks[drive]
 		else:


Test patch:
Repository: archlinux/archinstall

Issue:
gnome-extra provides WAY too much bloatware
I can't imagine most people wanting all the packages this installs on a new installation. Most of these applications are things like games and advanced tools like dconf-editor that your average user should not be touching. Some of them are nice to have but can be installed later manually instead of during initial installation.


Discussion:

Resolving patch:
diff --git a/profiles/applications/gnome.py b/profiles/applications/gnome.py
--- a/profiles/applications/gnome.py
+++ b/profiles/applications/gnome.py
@@ -1,4 +1,4 @@
 import archinstall
 
-installation.add_additional_packages("gnome gnome-extra gdm") # We'll create a gnome-minimal later, but for now, we'll avoid issues by giving more than we need.
-# Note: gdm should be part of the gnome group, but adding it here for clarity
\ No newline at end of file
+installation.add_additional_packages("gnome gnome-tweaks gnome-todo gnome-sound-recorder evolution gdm")
+# Note: gdm should be part of the gnome group, but adding it here for clarity


Test patch:
Repository: archlinux/archinstall

Issue:
Look in to enabling SMART for drives that support it
Something like `smartctl --smart=on --offlineauto=on --saveauto=on /dev/sda` where `archinstall.hardware.detectSmart()` finds drives that support it (to extend drive lifetime if possible).


Discussion:
These are the steps we use on Arch servers: https://gitlab.archlinux.org/archlinux/infrastructure/-/blob/master/roles/common/tasks/main.yml#L25

Resolving patch:
diff --git a/profiles/awesome.py b/profiles/awesome.py
--- a/profiles/awesome.py
+++ b/profiles/awesome.py
@@ -6,7 +6,7 @@
 
 # New way of defining packages for a profile, which is iterable and can be used out side
 # of the profile to get a list of "what packages will be installed".
-__packages__ = ['nano', 'nemo', 'gpicview-gtk3', 'openssh', 'sshfs', 'htop', 'scrot', 'wget']
+__packages__ = ['nemo', 'gpicview-gtk3', 'scrot']
 
 def _prep_function(*args, **kwargs):
 	"""
@@ -33,13 +33,7 @@ def _prep_function(*args, **kwargs):
 	awesome = archinstall.Application(installation, 'awesome')
 	awesome.install()
 
-	# Then setup and configure the desktop environment: awesome
-	editor = "nano"
-	filebrowser = "nemo gpicview-gtk3"
-	utils = "openssh sshfs htop scrot wget"
-
-
-	installation.add_additional_packages(f"{utils} {filebrowser} {editor}")
+	installation.add_additional_packages(__packages__)
 
 	alacritty = archinstall.Application(installation, 'alacritty')
 	alacritty.install()
diff --git a/profiles/desktop.py b/profiles/desktop.py
--- a/profiles/desktop.py
+++ b/profiles/desktop.py
@@ -4,6 +4,10 @@
 
 is_top_level_profile = True
 
+# New way of defining packages for a profile, which is iterable and can be used out side
+# of the profile to get a list of "what packages will be installed".
+__packages__ = ['nano', 'openssh', 'htop', 'wget', 'iwd', 'wireless_tools', 'wpa_supplicant', 'smartmontools']
+
 def _prep_function(*args, **kwargs):
 	"""
 	Magic function called by the importing installer
@@ -14,7 +18,7 @@ def _prep_function(*args, **kwargs):
 
 	supported_desktops = ['gnome', 'kde', 'awesome']
 	desktop = archinstall.generic_select(supported_desktops, 'Select your desired desktop environment: ')
-
+	
 	# Temporarily store the selected desktop profile
 	# in a session-safe location, since this module will get reloaded
 	# the next time it gets executed.
@@ -41,7 +45,11 @@ def _prep_function(*args, **kwargs):
 	There are plenty of desktop-turn-key-solutions based on Arch Linux,
 	this is therefore just a helper to get started
 	"""
+	
+	# Install common packages for all desktop environments
+	installation.add_additional_packages(__packages__)
 
 	# TODO: Remove magic variable 'installation' and place it
 	#       in archinstall.storage or archinstall.session/archinstall.installation
 	installation.install_profile(archinstall.storage['_desktop_profile'])
+


Test patch:
Repository: archlinux/archinstall

Issue:
AttributeError: module 'pathlib' has no attribute 'Paht'
This is a serious typo in the code of luks.py

I tried to create an encrypted luks partitiom but end up with this error in the title

Please fix as soon as possible



Discussion:
![IMG_20210408_194641_078.jpg](https://user-images.githubusercontent.com/81270448/114121555-68c89880-98a3-11eb-9e2d-a194f104678d.jpg)

Resolving patch:
diff --git a/archinstall/lib/luks.py b/archinstall/lib/luks.py
--- a/archinstall/lib/luks.py
+++ b/archinstall/lib/luks.py
@@ -127,7 +127,7 @@ def unlock(self, partition, mountpoint, key_file):
 			os.path.basename(mountpoint)  # TODO: Raise exception instead?
 
 		wait_timer = time.time()
-		while pathlib.Paht(partition.path).exists() is False and time.time() - wait_timer < 10:
+		while pathlib.Path(partition.path).exists() is False and time.time() - wait_timer < 10:
 			time.sleep(0.025)
 
 		sys_command(f'/usr/bin/cryptsetup open {partition.path} {mountpoint} --key-file {os.path.abspath(key_file)} --type luks2')


Test patch:
Repository: archlinux/archinstall

Issue:
Awesome profile installation failed with no such file or directory on xdg-mime
![VirtualBox_arch_09_04_2021_17_47_40](https://user-images.githubusercontent.com/277927/114245038-9f62e980-995d-11eb-965c-e5a9fbb9ded2.png)
![VirtualBox_arch_09_04_2021_17_53_17](https://user-images.githubusercontent.com/277927/114245039-9ffb8000-995d-11eb-8f73-2ece536d1c2a.png)

Resolve #261 and related issues
Closes #262.

🚨 PR Guidelines:

# New features *(v2.2.0)*

Merge new features in to `torxed-v2.2.0`.<br>
This branch is designated for potential breaking changes, added complexity and new functionality.

# Bug fixes *(v2.1.4)*

Merge against `master` for bug fixes and anything that improves stability and quality of life.<br>
This excludes:
 * New functionality
 * Added complexity
 * Breaking changes

Any changes to `master` automatically gets pulled in to `torxed-v2.2.0` to avoid merge hell.

# Describe your PR

If the changes has been discussed in an Issue, please tag it so we can backtrace from the Issue later on.<br>
If the PR is larger than ~20 lines, please describe it here unless described in an issue.

# Testing

Any new feature or stability improvement should be tested if possible.
Please follow the test instructions at the bottom of the README.

*These PR guidelines will change after 2021-05-01, which is when `v2.1.4` gets onto the new ISO*



Discussion:
Possible solution: add extra/xdg-utils to desktop package


Resolving patch:
diff --git a/profiles/desktop.py b/profiles/desktop.py
--- a/profiles/desktop.py
+++ b/profiles/desktop.py
@@ -6,7 +6,7 @@
 
 # New way of defining packages for a profile, which is iterable and can be used out side
 # of the profile to get a list of "what packages will be installed".
-__packages__ = ['nano', 'vim', 'openssh', 'htop', 'wget', 'iwd', 'wireless_tools', 'wpa_supplicant', 'smartmontools']
+__packages__ = ['nano', 'vim', 'openssh', 'htop', 'wget', 'iwd', 'wireless_tools', 'wpa_supplicant', 'smartmontools', 'xdg-utils']
 
 def _prep_function(*args, **kwargs):
 	"""


Test patch:
Repository: archlinux/archinstall

Issue:
Why is argon2i used?
Referring to: https://github.com/archlinux/archinstall/blob/master/archinstall/lib/luks.py#L73

I read that argon2id gives better security, so is there any reason against it?


Discussion:
I'm not sure if this [discussion #154 *(Why use Argon2i over aes-xts-plain64?)*](https://github.com/archlinux/archinstall/discussions/154) answers this?
I think it's a trade off to not sacrifice speed potential.
I was more asking of why `argon2i` not `argon2id`. That is not so clear to me still. I dont think we need too much configuration here, but a good default. To me it seems that `id` is better than just `i`, but I might be wrong with that assumption.
> I was more asking of why `argon2i` not `argon2id`. That is not so clear to me still. I dont think we need too much configuration here, but a good default. To me it seems that `id` is better than just `i`, but I might be wrong with that assumption.

Other than rumors about performance impact and never have tested it, there's no reason why not. Would be a welcome change if we can compare some benchmarks and test for stability.
From Wikipedia, 

The Internet draft recommends using Argon2id except when there are reasons to prefer one of the other two modes.

As such, +1 to switching it. 

Resolving patch:
diff --git a/archinstall/lib/luks.py b/archinstall/lib/luks.py
--- a/archinstall/lib/luks.py
+++ b/archinstall/lib/luks.py
@@ -70,7 +70,7 @@ def encrypt(self, partition, password=None, key_size=512, hash_type='sha512', it
 			'--batch-mode',
 			'--verbose',
 			'--type', 'luks2',
-			'--pbkdf', 'argon2i',
+			'--pbkdf', 'argon2id',
 			'--hash', hash_type,
 			'--key-size', str(key_size),
 			'--iter-time', str(iter_time),


Test patch:
Repository: archlinux/archinstall

Issue:
Default timezone to UTC
We have defaults for locale. For timezone, UTC is a great default as it's VERY common to use time timezone for servers.




Discussion:

Resolving patch:
diff --git a/archinstall/lib/user_interaction.py b/archinstall/lib/user_interaction.py
--- a/archinstall/lib/user_interaction.py
+++ b/archinstall/lib/user_interaction.py
@@ -130,7 +130,9 @@ def ask_for_additional_users(prompt='Any additional users to install (leave blan
 	return users, super_users
 
 def ask_for_a_timezone():
-	timezone = input('Enter a valid timezone (Example: Europe/Stockholm): ').strip()
+	timezone = input('Enter a valid timezone (examples: Europe/Stockholm, US/Eastern) or press enter to use UTC: ').strip()
+	if timezone == '':
+		timezone = 'UTC'
 	if (pathlib.Path("/usr")/"share"/"zoneinfo"/timezone).exists():
 		return timezone
 	else:


Test patch:
Repository: archlinux/archinstall

Issue:
Add option to select shell
When creating a new user, archinstall should ask, which shell (bash,zsh,ksh,fish) the user wants to use.


Discussion:
It wouldn't be hard to do, but it might be a bad idea to overwhelm users with questions and you can always get an alternate shell anyways with adding it in additional packages and running chsh. I feel like this is easy enough to do post-install (I use ohmyzsh myself), that it's not a huge deal to have it be in the installer itself.
A good compromise would be making a change_shell method that takes in the desired shell and user as arguments and allow it to be called when using the library, not the guided installer. Thoughts, @Torxed ? 

Resolving patch:
diff --git a/archinstall/lib/installer.py b/archinstall/lib/installer.py
--- a/archinstall/lib/installer.py
+++ b/archinstall/lib/installer.py
@@ -443,6 +443,12 @@ def user_set_pw(self, user, password):
 
 		o = b''.join(sys_command(f"/usr/bin/arch-chroot {self.target} sh -c \"echo '{user}:{password}' | chpasswd\""))
 		pass
+						  
+	def user_set_shell(self, user, shell):
+		self.log(f'Setting shell for {user} to {shell}', level=LOG_LEVELS.Info)
+
+		o = b''.join(sys_command(f"/usr/bin/arch-chroot {self.target} sh -c \"chsh -s {shell} {user}\""))
+		pass
 
 	def set_keyboard_language(self, language):
 		if len(language.strip()):


Test patch:
Repository: archlinux/archinstall

Issue:
Timezone field does not have validation in latest master (2.1.4)
Entering, for instance, US/Easter does not provide an error and opportunity to enter a valid result as intended (in this case US/Eastern). This results in a null timezone in the config that is installed.


Discussion:

Resolving patch:
diff --git a/archinstall/lib/user_interaction.py b/archinstall/lib/user_interaction.py
--- a/archinstall/lib/user_interaction.py
+++ b/archinstall/lib/user_interaction.py
@@ -130,17 +130,18 @@ def ask_for_additional_users(prompt='Any additional users to install (leave blan
 	return users, super_users
 
 def ask_for_a_timezone():
-	timezone = input('Enter a valid timezone (examples: Europe/Stockholm, US/Eastern) or press enter to use UTC: ').strip()
-	if timezone == '':
-		timezone = 'UTC'
-	if (pathlib.Path("/usr")/"share"/"zoneinfo"/timezone).exists():
-		return timezone
-	else:
-		log(
-			f"Time zone {timezone} does not exist, continuing with system default.",
-			level=LOG_LEVELS.Warning,
-			fg='red'
-		)
+	while True:
+		timezone = input('Enter a valid timezone (examples: Europe/Stockholm, US/Eastern) or press enter to use UTC: ').strip()
+		if timezone == '':
+			timezone = 'UTC'
+		if (pathlib.Path("/usr")/"share"/"zoneinfo"/timezone).exists():
+			return timezone
+		else:
+			log(
+				f"Specified timezone {timezone} does not exist.",
+				level=LOG_LEVELS.Warning,
+				fg='red'
+			)
 		
 def ask_for_audio_selection():
 	audio = "pulseaudio" # Default for most desktop environments


Test patch:
Repository: archlinux/archinstall

Issue:
Formatting fails under certain conditions
### Description:
There is an issue where formatting may fail if user skips manual partitioning step.
**Example:**

![image](https://user-images.githubusercontent.com/36604233/115281797-11071880-a152-11eb-93b6-373ffc89513d.png)

### Steps to reproduce:
1. Use `archinstall` and fully install system with it.
2. Run `archinstall` again.
3. Select the drive and select "Keep existing partitions ...".
4. When you get a list of partitions, press Enter and continue with the installation.
5. After the formatting countdown, formatting will fail.

> Tested with latest `archinstall` master branch changes on VirtualBox 6.1


Discussion:
Ah, ofc we need a verification that the user selected at least two partitions, `/` and `/boot`!
I wondering, what partition do it set by default. Because, it fails even when I select any partition and just hit Enter twice
Nothing should be set by default, if "keep partitions" is selected, there has to be a user choice.
The installer will not try to make any informed decisions based on the disks themselves.
It might be worthwhile to check for `/mnt` and `/mnt/boot` and replace them with `/` and `/boot` *(or leave as is)*
In guided we can probably do some magic and replace `/mnt` with `/` for instance, since I don't see any use case where they would ever need to actually write `/mnt` unless they chose "Use /mnt as is" which is the third new option. But I'd like to merge your other PR before starting on fixing this since it has to do with user-input :)

Resolving patch:
diff --git a/examples/guided.py b/examples/guided.py
--- a/examples/guided.py
+++ b/examples/guided.py
@@ -74,6 +74,7 @@ def ask_user_questions():
 
 			archinstall.log(f" ** You will now select which partitions to use by selecting mount points (inside the installation). **")
 			archinstall.log(f" ** The root would be a simple / and the boot partition /boot (as all paths are relative inside the installation). **")
+			mountpoints_set = []
 			while True:
 				# Select a partition
 				# If we provide keys as options, it's better to convert them to list and sort before passing
@@ -81,7 +82,10 @@ def ask_user_questions():
 				partition = archinstall.generic_select(mountpoints_list,
 														"Select a partition by number that you want to set a mount-point for (leave blank when done): ")
 				if not partition:
-					break
+					if set(mountpoints_set) & {'/', '/boot'} == {'/', '/boot'}:
+						break
+
+					continue
 
 				# Select a mount-point
 				mountpoint = input(f"Enter a mount-point for {partition}: ").strip(' ')
@@ -122,6 +126,7 @@ def ask_user_questions():
 					# We can safely mark the partition for formatting and where to mount it.
 					# TODO: allow_formatting might be redundant since target_mountpoint should only be
 					#       set if we actually want to format it anyway.
+					mountpoints_set.append(mountpoint)
 					partition.allow_formatting = True
 					partition.target_mountpoint = mountpoint
 					# Only overwrite the filesystem definition if we selected one:


Test patch:
Repository: archlinux/archinstall

Issue:
_gfx_driver_packages not defined when choosing sway option (current master build)
![VirtualBox_Arch_30_04_2021_23_01_50](https://user-images.githubusercontent.com/277927/116769175-37f20400-aa08-11eb-92cb-6c50348296d1.png)

_gfx_driver_packages not defined when choosing sway option (current master build)
![VirtualBox_Arch_30_04_2021_23_01_50](https://user-images.githubusercontent.com/277927/116769175-37f20400-aa08-11eb-92cb-6c50348296d1.png)

_gfx_driver_packages not defined when choosing sway option (current master build)
![VirtualBox_Arch_30_04_2021_23_01_50](https://user-images.githubusercontent.com/277927/116769175-37f20400-aa08-11eb-92cb-6c50348296d1.png)



Discussion:

Resolving patch:
diff --git a/profiles/sway.py b/profiles/sway.py
--- a/profiles/sway.py
+++ b/profiles/sway.py
@@ -4,7 +4,19 @@
 
 is_top_level_profile = False
 
-__packages__ = ["sway", "swaylock", "swayidle", "waybar", "dmenu", "light", "grim", "slurp", "pavucontrol", "alacritty"]
+__packages__ = [
+	"sway",
+	"swaylock",
+	"swayidle",
+	"waybar",
+	"dmenu",
+	"light",
+	"grim",
+	"slurp",
+	"pavucontrol",
+	"alacritty",
+]
+
 
 def _prep_function(*args, **kwargs):
 	"""
@@ -13,18 +25,26 @@ def _prep_function(*args, **kwargs):
 	other code in this stage. So it's a safe way to ask the user
 	for more input before any other installer steps start.
 	"""
-	if "nvidia" in _gfx_driver_packages:
-		choice = input("The proprietary Nvidia driver is not supported by Sway. It is likely that you will run into issues. Continue anyways? [y/N] ")
-		if choice.lower() in ("n", ""):
-			raise archinstall.lib.exceptions.HardwareIncompatibilityError("Sway does not support the proprietary nvidia drivers.")
-
-	__builtins__['_gfx_driver_packages'] = archinstall.select_driver()
+	__builtins__["_gfx_driver_packages"] = archinstall.select_driver()
 
 	return True
 
+
 # Ensures that this code only gets executed if executed
 # through importlib.util.spec_from_file_location("sway", "/somewhere/sway.py")
 # or through conventional import sway
-if __name__ == 'sway':
+if __name__ == "sway":
+	if "nvidia" in _gfx_driver_packages:
+		choice = input(
+			"The proprietary Nvidia driver is not supported by Sway. It is likely that you will run into issues. Continue anyways? [y/N] "
+		)
+		if choice.lower() in ("n", ""):
+			raise archinstall.lib.exceptions.HardwareIncompatibilityError(
+				"Sway does not support the proprietary nvidia drivers."
+			)
+
 	# Install the Sway packages
 	installation.add_additional_packages(__packages__)
+
+	# Install the graphics driver packages
+	installation.add_additional_packages(_gfx_driver_packages)


Test patch: