Ubuntu GParted Question - Page 1
Forum section: electronics / general

author: bostonman
author_rank: Super Contributor
timestamp: February 25, 2021, 03:26:33 pm
content: When I install a new drive in a Windows based PC (although I haven't for quite some time) it's usually just a simple format.In Ubuntu I'm getting confused. Ubuntu doesn't recognize a blank (new) drive, so I figured out to use GParted. I've tried creating a new partition, but then also there is the option to create a partition table under device. Doing both doesn't seem to have an advantage, so I just create a partition (under partition drop down menu).At this point Ubuntu doesn't see the new drive, so I have to unplug and reconnect it (sometimes more than once), and then it's finally recognized (a bit of annoying process).After I use the command: dd if=file.img of=/dev/sdc status=progressThe end result (see attached image) looks as if the image is on one partition and a second partition exists. If I'm understanding correctly, this would mean I can't write any files to the partition that is 9.36GB in size because it's maxed, is this correct?My assumption was that after I burn the image, I should have one 111.79GB long partition with only 9.36GB occupied.

author: grumpydoc
author_rank: Super Contributor
timestamp: February 25, 2021, 03:33:22 pm
content: Is this a USB drive or an internal drive?Your dd command will destroy whatever partition table you set up, but I'm not clear why you are trying to initialise the new disk from an image file.You only have one partition on the disk which is 9.36G in size, the rest is unallocated.

author: bostonman
author_rank: Super Contributor
timestamp: February 25, 2021, 03:38:04 pm
content: It's going to be a bootable drive (for an oscilloscope).The image is a full Win98 OS, and the drive is an SSD connected via a USB convertor so I can burn the image.I know what a partition is, but with Windows, I've never needed to create one within reason (I have to put Ubuntu as a secondary OS). With Windows, I basically format a X GB drive (let's call it 100GB), and I have 100GB worth of empty space.From what I interpret, I have two partitions, which, in Windows, would be two separate drives. If so, then the 9.36GB partition can't have anything added to it.

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 03:40:02 pm
content: Quote from: bostonman on February 25, 2021, 03:26:33 pmThere is no possible comparison between MS WIN and *NIX.The number of choices on *NIX are just orders of magnitude above.Gparted is handy for several things.. (like resizing etc)But definitively no *NIX folk do blank disk installations using that..The two tools for proper partition definition and alignment areFDISK for MBR disk typesGDISK for GPT disk types.ABOVE 1.5G you are required to use GPTotherwise MBRs are still a good choice for compatible reasons.Just define a single partition using a console with one of thesetools and no problems should arise...Format that with a proper FS ..If the the OS itself did not perform a DISK SYNCH..all tools come with a SYNCH option just after the write operationwhich can be evoked any time by console..Advice that this is dangerous stuff...** SAFETY WARNING **DO NOT ISSUE A WRITE WITHOUT 100% SURE..kinda shooting your own foot... it happensPaul

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 03:41:40 pm
content: Quote from: bostonman on February 25, 2021, 03:38:04 pmUSE THE MBR FDISK based tool..It is the proper tool and only that for such OSDo not forget to place a proper MBR on the bottable diskand flag that as "bottable"Paul

author: bostonman
author_rank: Super Contributor
timestamp: February 25, 2021, 03:45:24 pm
content: Okay, I'll try this tonight.As for my assumption, this is two partitions with one being limited to just 9.36GB? Once installed in a Windows based system, it will show two drives, correct?

author: grumpydoc
author_rank: Super Contributor
timestamp: February 25, 2021, 04:01:42 pm
content: Quote from: bostonman on February 25, 2021, 03:45:24 pmIgnore PKTKS' advice for a sec - not that it's wrong in the more general sense but I think that it is wrong in this case, or not the whole story.It looks like the image file is, as you say, the whole disk not just the C: partition, so itincludesthe partition table. It does not much matter that it was originally a smaller disk - you get the original partition and a whole load of empty space.You *could* enlarge that partition to cover the whole disk (W98 has a 128GB limit per disk which you are just inside) but I advise against trying to do that on a bootable image with Linux tools. The W98 boot process is sensitive to the exact location/size of some disk structures - I remember digging out exactly which bits to edit, but it was always a bit hit and miss as I recall. Better to use one of the commercial partition editors or the Windows OS itself (though I don't recall if W98 is smart emough to re-size an in-use C: partition)If you just want to format the unallocated space as the D: drive in windows that should be straightforward - select the unallocated space, add a FAT32/VFAT partition occupying the whole of the free space and format it with FAT32.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 25, 2021, 04:09:12 pm
content: The/dev/sdcblock device represents the entire drive. When you write to it (like you did withdd), you overwrite the contents of the drive, including any partition table and at least initial partitions (if any).A block device can, but does not have to, contain partitions. Even USB sticks can have partitions. (If you use a DOS type partition table, aka MBR or Master Boot Record, and FAT32/VFAT file systems on each partition, all that happens is that when you stick the USB stick in, you'll see more than one logical drive appear.)We do not know what yourfile.imgcontains. You can associate the image file with a loopback device usingsudo losetup -fP file.imgwhere the -f flag tells losetup to use first free loopback device (it'll tell you what it is, say/dev/loop3), and -P flag to also scan its partition table.After this, you can run gparted, pointing it to that particular loopback device, and modify the partitions, even add new ones. To ensure you use the right device, runsudo gparted /dev/loop3(or whatever loopback device losetup gave you).After you have made changes (they are written back to the image file, so do keep a backup copy!), detach the loop device, and reattach (may give you a new loopback device!) usingsudo losetup -d /dev/loop3sudo losetup -fP file.imgso that the new partition table is guaranteed to be in use.You can format any new partitions you create; the new partitions will appear as/dev/loopNpP, whereNis the loop device number (output by the losetup command), andPis the partition number.When you're done, runsudo syncsudo losetup -d /dev/loopN(Thesyncis just an extra precaution, and makes sure all modified files – including the image file – are actually written and not just cached in RAM.In this case, you have already written the disk image to/dev/sdc. (Nothing you did before thatddcommand matters.)So, why not edit the disk partition table afterwards, using say gparted? Just make sure you don't have any of its partitions mounted when you do.What grumpydoc said is absolutely true: moving the existing partition may make the disk unbootable. (See e.g.gparted manual for moving partitionsandfixing system boot problems.)

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 04:24:07 pm
content: please make it plain simple...absolute trivial...My old stuff is crammed in NBD virtual disksnevertheless the procedure is _E_X_A_C_T _ the very samefdisk /dev/sdxmake a single partition with bottable flagAttached my NBD virtual disk MBRPaul

author: grumpydoc
author_rank: Super Contributor
timestamp: February 25, 2021, 04:42:32 pm
content: Quote from: PKTKS on February 25, 2021, 04:24:07 pmYes, but from what bostonman has said, he has a disk image, containing a bootable copy of Windows 98 being the operating system and firmware for an oscilloscope - with which he needs/wants to initialise his new disk.To do so just the "dd" command is sufficient (though I'd add a "sync" afterwards), he should be able to move the new disk straight to the 'scope and it will boot.IF he wants to make the existing partition bigger, then the linux tools will do that but probably destry the ability to boot off the disk - so it is safer to use Windows tools for that.

author: PKTKS
author_rank: Super Contributor
timestamp: February 26, 2021, 05:10:26 pm
content: ... my first understanding was that he tried that before..And the result was that misaligned partition...Due to the evolution of disk geometries... i HAD TONS ..of such misaligned partitions when just trying to dd someold image to new disks ...Bottom line is that the dd image may boot .. or notdepending of factors like new disk tech (geometry) andother factors..My only one safe advice is:- DO THE PARTITIONING THE ABSOLUTE PROPER WAY:fdisk /dev/sdX- save that and format the disk using FAT32 limits - bootable flag!!!- *UNIX does not care about bootable flag at all but MS will not boot wo that.- mount your old image in whatever means possible..- that depends on the applet used to image the file system.- copy the file system over the new formatted partition..then... the real safe part... if that old stuff is 90s win9x...you need to:C:> SYS C: (or A:)to actually have a bootable drive..shitty as it always was there no safer method..but alternatives may be found... and enough to live with.last piece of desperate trying is converting that IMG to a virtual diskand booting that in a suited VM so you can extract the file systemand also create some kind of bootable image to run SYS on that..other methods exist to put a boot loader on the MBR which willfound and boot C:\io.sys directly ( SYSLINUX can do that )The very first version of GRUB can also do that:Code:find --set-root --ignore-floppies --ignore-cd /io.sysrootnoverifychainloader +1bootBingo grub boot that old thing magically wo SYS and DDPaul

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 26, 2021, 05:38:20 pm
content: Next step is you need to resize the partition so it uses the unallocated space. GParted can do this.

author: Monkeh
author_rank: Super Contributor
timestamp: February 26, 2021, 05:45:39 pm
content: Quote from: PKTKS on February 26, 2021, 05:10:26 pmWhat misaligned partition?

author: grumpydoc
author_rank: Super Contributor
timestamp: February 26, 2021, 05:45:49 pm
content: Quote from: PKTKS on February 26, 2021, 05:10:26 pmThe partition was not "misaligned" - it just didn't occupy the whole disk.Which is what you'd expect when copying the whole disk image from a smaller disk to a larger one.QuoteYes, back in the mists of time when disks were read using C/H/S (Int 13 function 2/3) addresses you had to be much more careful - but logical block addressing has been the norm for, what?, 25 years now at least.QuoteOK, yes - you are right about the SYS command.But you need to have W98 booted to use SYS to copy the boot files over to a new drive - which presents you with a catch-22 situation....QuoteYou can create a bootable DOS disk that way, but Windows hasn't booted via DOS since 3.11 days, just copying the files to a freshly formatted disk is not sufficient - there is some special sauce (I did know how to do it waaaay back but would have to dig around and see if Google can refresh my memory and life is almost certainly too short).However Iwouldexpect a disk image which has been created with dd to boot just fine, however the partition will not automatically fill a new, larger, disk.Bostonman was incorrect in his original assumption that transferring the old image with dd would leave him with a large ~110Gb partition with 9G or so used. What he got is a 9Gb partition which was full and ~100Gb of unallocated space on the disk - but that's what you would expect.I'd suggest the following steps, assuming that the target 'scope is available.dd the image as described, follow that with a sync command to flush the blocks to disk and transfer to the 'scopeif it boots, greatif not you've got other problems like C/H/S *does* matter to the 'scope or a corrupt image and you *will* need to figure out how to manually copy the files and enable the Windows boot process, or you have a corrupt imageShut the scope down and put it back in the Ubuntu machine, use Gparted to enlarge the 9Gb partition to fill the diskPut it back in the 'scope, if it boots then you're home and dryIf not then Gparted moved some critical files. IIRC the real issue was moving a partition but Google should be able to cough up the details of how to patch the image.Quote from: SiliconWizard on February 26, 2021, 05:38:20 pmYes, it can though if any boot files need to be in a fixed spot on the disk and they get moved it might not boot - that said, thinking about it for a fraction longer, ISTR that the big issue was moving the start of a partition - there's an offset stashed somewhere iin the early boot loader which needs updating.

author: PKTKS
author_rank: Super Contributor
timestamp: February 26, 2021, 05:51:05 pm
content: Quote from: SiliconWizard on February 26, 2021, 05:38:20 pmnot always...sometimes (very frequently) gparted will just not be able tohandle that "phantom" space or misalignment that you willonly see there as well...doing the partition table on fdisk will never show you those phantoms.seems a gparted exclusive "feature" ...But Mileage may vary...Paul

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 27, 2021, 12:02:13 am
content: I've never seen that happen. Unless maybe the copied partition is a mess to begin with.

author: PKTKS
author_rank: Super Contributor
timestamp: February 27, 2021, 12:33:43 pm
content: Quote from: SiliconWizard on February 27, 2021, 12:02:13 amNot necessarily...As said above old geometry having different "boundaries"when just "dd" over a device with different C/H/S **MAY** hitmisaligned boundaries on the partition end (if aligned byMeg or by cylinder) or.. show these small phantoms betweenpartitionsthere is no possible way to recover that discrepancies in usablespace unless doing a new partition schemaIt is not about "free space" ... the partition boundaries do not alignexactly between devices..It is harmless but very annoying when presented..Paul

author: bostonman
author_rank: Super Contributor
timestamp: February 27, 2021, 02:42:51 pm
content: Sorry I've been quiet, I got sidetracked with an issue and haven't had time to try any suggestions.I'll need to read through this and try the suggestions tomorrow.As for my intentions, I didn't realize this was such a loaded question. The file I have was compressed in Linux, so it's one reason I've been working in Linux. I thought the process would be as simple as burning the image onto any hard drive (providing it has the minimum size to fit the image), and the image would over write the entire drive (within reason) leaving me with a bootable drive and remaining empty space (in this case, rounding up, and using ideal numbers, 120GB - 10GB, or 110GB).Past experiences with using Gparted was that I'd burn the image, and then the drive wouldn't boot because the image got burned onto another partition (or maybe it wasn't flagged as bootable). After basically getting lucky, I'd burn the same image and the drive would successfully boot.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 27, 2021, 03:48:23 pm
content: Quote from: bostonman on February 27, 2021, 02:42:51 pmNo, unfortunately: the image also specifies the size of the file system on the drive.You should be able to just resize the file system afterwards using many tools, gparted included. This should not cause the file system to become unbootable. (It can occur if the partition is not aligned to a 4k boundary, and the tools move it.) You can also use Windows tools to resize the partition, although I don't use Windows so I cannot suggest any particular tool.If the image contains an extended partition (it will be marked as such – it is a container for secondary partitions, MBR partition table having room for only four primary partitions), then you do need to resize the extended partition first to the full drive size, before you can extend the secondary partitions inside it.

author: bostonman
author_rank: Super Contributor
timestamp: February 28, 2021, 04:42:36 pm
content: QuoteThe problem I initially had: the hard drive was fresh out of the box. Simply connecting it to Ubuntu (SATA to USB converter) didn't show the drive. I'll admit my computer experience never dealt with a "blank" drive. If anything, I'd boot a PC with a Windows CD and install Windows that way. Maybe back in the DOS days I dealt with a fresh drive, however, I don't remember, and things have changed.In any case, I searched online and learned about Gparted. I thought once I format the drive, that would allow (in this case) DD to copy the image thus making it a normal bootable drive. I don't remember the steps, but at some point the drop down menu wouldn't be highlighted to format, other times I'd have to create a partition before formatting, etc... Afterwards, when I finally had what I thought was a formatted drive, I couldn't see it in Terminal.I'd unplug the USB and reconnect it, and, for whatever reason, eventually the drive would appear with some odd name whereas in Windows I could name the drive.Anyway, those were the steps I took. Once I saw a 10GB partition and a separate 110GB partition, I "assumed" that it's A: two separate drives since it's two partitions, and B: I couldn't write anything to the 10GB partition because the extracted image occupies exactly the size it needed.Reading through all the advice, it seems the anomaly is that I'm trying to burn a bootable image and have the remaining space available without taking extra steps.I guess the steps I should take (from what I interpreted) is to burn the image as usual, attempt to resize the partition thus making the other partition smaller, and still ending up with two partitions, only the first (bootable) one will be resized.

author: Monkeh
author_rank: Super Contributor
timestamp: February 28, 2021, 04:57:18 pm
content: Okay, so there's a lot of basic misunderstanding going on here.Quote from: bostonman on February 28, 2021, 04:42:36 pmThis is because it's an unpartitioned drive with nothing on it - there's nothing for unprivileged tools to do with it. The kernel log ('dmesg') will show you a nice long string of messages culminating in the detection of a new block device.QuoteCompletely unnecessary because your image includes the partition table.QuoteYou only have one partition. The rest of the space is unallocated. What you have is an exact copy of the original drive contents plopped into a larger space.All you need to do is write the image to the raw device (/dev/sdx, where x is the correct device and not the wrong device), pull the drive, and put it where it belongs. Everything should work fine. If you need to enlarge the partition instead of simply adding a second one (which you can do from Windows..), well, I leave advice on that to people familiar with working on 20 year old half-functional OSes.Most importantly in all of this: Don't listen to PKTKS. He's not paying attention and will insist on leading you on wild goose chases to satisfy his strange urges.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 28, 2021, 06:28:28 pm
content: There is nothing wrong in Monkeh's suggestion, but here's mine:Runlsblkorbash -c '(cd /sys/block && for D in */device/model ; do echo -n "${D%%/*}: " ; cat "$D" ; done)'to determine which device is the correct target device. For SATA and USB drives, it will be/dev/sdX, for NVMe drives it will be/dev/nvmeXnY. For simplicity, I will use/dev/sdXfor the device name. Partitions will be named (numbered)/dev/sdXPand/dev/nvmeXnYpP, respectively.Write the image over the drive using e.g.sudo dd if=image.img of=/dev/sdX bs=2M status=progressRemember that this will destroy any partition tables and possibly any data on the drive. Thebs=2Mis just an optimization, telling dd to use two megabyte chunks, as larger block sizes like this tend to yield optimum I/O speeds. The final block may be smaller; the image does not have to be a multiple of two megabytes in size.Optionally, make sure the data is written to the device, and not just cached in RAM, by runningsudo syncand make sure none of the partitions on it are mounted, by runningsudo umount /dev/sdX*Usually, the drive now has only one partition, which does not cover the entire drive.Use your favourite partition manager program to resize it, so it covers the entire drive. For example,sudo gparted /dev/sdXDo not try tomovethe partition (change itsstart), only resize (change itsendorsize) the existing one to cover the entire disk. If asked, decline moving the partition. Moving the partition is what usually causes the drive to become unbootable in MS-DOS or Windows; resizing should not affect bootability.Unused areaorunpartitioned areashown in the partition manager program refers to the part of the disk that is not covered by any partitions. It cannot be mounted, and it does not show up as a separate drive in DOS/Windows. It is just unallocated, unused space on the drive.If you have multiple partitions, you may have anExtended partition. This is a container in MBR partitioning scheme, and must be resized to fill the rest of the drive before any partitions within it can be extended to cover the entire drive. Some partition manager programs do this automatically for you, others limit the size of partitions within it to the size of the extended partition.Runsudo syncto ensure changes are written to the device, andsudo partx --add /dev/sdXto ensure the kernel knows about the partitions on that device. The latter may cause the partition(s) to be automounted, so before detaching the device, unmount the partitions either via your desktop (ejecting the drive), or by runningsudo umount /dev/sdX*You can now either mount the device (if not automounted in the previous step), or detach it (but do check your desktop if it automounted it; do not detach mounted drives, as that can cause the data on the drive to become garbled) and attach to the target machine.

author: PKTKS
author_rank: Super Contributor
timestamp: March 01, 2021, 09:23:48 am
content: My advice is simple:- IF IT JUST **WORKS** than go with it..You can try just DD and see.. if it boots keep it..if not follow the advices...#Nominal placed some very good tips and explanationsabout the file system boundary results on the partition...If it just works go with it.. if not follow the tips..Alas there is no hidden agenda whatsoever as patronized (!$#_)Paul

author: bostonman
author_rank: Super Contributor
timestamp: March 02, 2021, 02:08:24 am
content: QuoteI followed your steps but forgot to do the above. In any case, the drive still had two partitions, or maybe one wasn't a partition, but simply unallocated. I resized the bootable partition (the end of the partition, the ride side, not the beginning), and it made a single line (i.e. one) partition after I maxed out the right side.After I connected it in the scope (via a IDE to SATA converter that I just bought) and the scope booted. I took a picture of the disk space for reference (see attached). Sadly this is a waste of a drive because I'm not writing 120GB of data to it, however, it wasn't expensive. Any smaller sized drives which seemed hard to find weren't much cheaper.

author: PKTKS
author_rank: Super Contributor
timestamp: March 02, 2021, 10:20:42 am
content: back to the points...There are 2 things (distinct ones):- the partition size- the file system size.They MAY not be measured with same metrics...While one is based on C/H/S and LBA converted tablesthe other is measured is bits/bytes and Meg/Gig (aka base 2)...When properly done they align..when done like this case.. using DD over a new device...there may be problems in alignment...You first extend the partition boundary to the right... OKthis seems already done..Then you need to extend the file system size to the (NEW) partition boundaryHere is the point of problems..The tool you use is different and it will need to rebuild thewhole allocation table of your files...There are chances of serious problems.. or may be not.Using a well written software it should be OKand *MAY BE* just a small phantom arises at the end of the boundary.May be not..Paul

author: Nominal Animal
author_rank: Super Contributor
timestamp: March 02, 2021, 03:01:31 pm
content: Quote from: bostonman on March 02, 2021, 02:08:24 amNo harm done, I believe: Linux partition managers I know all do an fsync on the device (a more targeted sync), so the sync command is optional. And if you detached the drive, there was no reason for the Linux kernel to know about the partitions anyway. (The reason I included partx in the steps is that if one uses it, one does not need to detach and reattach the drive.) And because the modified partition didn't get automounted, there was no need to unmount anything.Happy to see the drive booted in Windows, and is now working; thanks for the screenshot!

author: bostonman
author_rank: Super Contributor
timestamp: March 02, 2021, 03:58:24 pm
content: Thanks for all the help!I’m guessing since it booted that I won’t get any odd behavior at some point later, so I’m assuming this has been resolved.

author: PKTKS
author_rank: Super Contributor
timestamp: March 02, 2021, 04:45:51 pm
content: Quote from: Nominal Animal on March 02, 2021, 03:01:31 pmthis is absolute righteach one syncs a target..