Restored VM Fails to Boot

[kevenw]

I'm having an issue with my mondorestore'ed VM failing to boot after the post-mondorestore reboot. I believe the bootloader (GRUB2) is not being setup correctly by mondorestore.

I've attached mondorestore.log, and I have 3 areas of concern: 1.

INFO: Found Device /dev/vda has a MBR partition type partition table format type on /dev/vda
DBG2: [Main] ../src/mondorestore/mondo-prep.c->partition_device#1539: partition_device('/dev/vda', 1, 0, 'xfs', 1048576) --- starting
DBG2: [Main] ../src/mondorestore/mondo-prep.c->partition_device#1554: Partitioning device /dev/vda1 (1024 MB)
INFO: Found Device /dev/vda has a MBR partition type partition table format type on /dev/vda
DBG1: [Main] ../src/mondorestore/mondo-prep.c->partition_device#1616: Doing the new all-in-one fdisk thing
DBG1: [Main] ../src/mondorestore/mondo-prep.c->partition_device#1617: output = 'n
p
1

+1048576K
'
DBG1: [Main] ../src/mondorestore/mondo-prep.c->partition_device#1624: last line = p
DBG1: [Main] ../src/mondorestore/mondo-prep.c->partition_device#1625: Failed to create partition 1 on /dev/vda; sending 'Enter'...

What would cause this?

2.

DBG2: [Main] ../src/mondorestore/mondo-rstr-tools.c->run_grub#1476: mr-grub /dev/vda /tmp/mountlist.txt
Now I'll use grub-install
Launching: chroot /mnt/RESTORING grub-install /dev/vda
chroot: can't execute 'grub-install': No such file or directory
grub-install returned 127
Now I'll use grub2-install
Installing for i386-pc platform.
grub2-install: error: cannot find a GRUB drive for /dev/vda1.  Check your device.map.
grub2-install returned 1

The device.map file has "dev/vda". I’ve also tried manually changing device.map to “/dev/vda1”, but I still see the same error. Any ideas why this might be occurring?

3.

Trying a hack
Checking usr/lib/grub/i386-pc
Cannot find BOOTPATHNAME
.............Cool.
	device (hd0) /dev/vda
	
	root (hd0,0)
	setup (hd0)
	quit 
chroot: can't execute 'grub': No such file or directory
Hack returned res=0

This is a bug in try_grub_hack() of mr-grub. If grub can't be found, the function returns a false positive.

[Bruno Cornec]

Outisde of the issue you're reporting, you're missing some dependencies:

/sbin/mindi: line 3071: mr-getparam: command not found

You should install the 3.3 version of perl-MondoRescue to get it. (​ftp://ftp.project-builder.org/test/rhel/7/x86_64/perl-MondoRescue-3.3.0-0.20190813231609.s3716.rhel7.noarch.rpm)

[Bruno Cornec]

On my side, using the latest 3.3.0 (20190813231609) I couldn't reproduce that issue anymore with a RHEL 7.4 version.

Please confirm whether you still have an issue on your side with that version.

[kevenw]

Still seeing issues 2 & 3, with mondo 3.3.0, on RHEL 7.6.

Still causing VM not to boot from Hard Disk, after mondorestore then reboot.

Attached new logs from attempt with 3.3.0.

[kevenw]

Is there any update on this ticket, or a timeline of when it can be looked at?

[Bruno Cornec]

I've looked again at your log files, and my test isn't similar to yours. One, I use QEMU, but I'm pretty sure it's unrelated. Two, you use XFS everywhere and I wasn't (IIRC) which may have an impact. Three, your error message says that: error: cannot find a GRUB drive for /dev/vda1

which in fact is true as your device.map file doesn't contain it.

I'm more suspecting this point for now. Once restored, and before rebooting the system, could you try to type from the shell: mr-mount-me (there will be errors for LVM volumes) Then you could chroot in the /mnt/RESTORING dir and

1/ try to type grub2-install /dev/vda (that may work alone which means that mondorestore doesn't use the correct device file for restoring the Boot loader) 2/ add a /dev/vda1 entry to your device.map file and then retype again grub2-install /dev/vda1 (that may work as well, which may mea that something removed that entry from your device.map, or that the installer didn't put it which I doubt).

Could you also attch your /boot/grub2/grub.cfg file please (it's missing from the current log, will be in next version - cf: [3722] I'd like to see what devices are used in it.

Resolution will be immediate (well theoriticall) once the diagnostic is correctly done, which isn't the case for now sorry for that.

[Bruno Cornec]

Point 2 is unrelated as I was also using XFS for all FS in my VM.

The difference of hypervisor could be an issue, as it presents my disk as /dev/sda not vda. So that may indeed create a difference. However my device.map doesn't reference sda1 either just sda as yours with vda.

In your case the command launched in the chrooot to restore the Boot Loader should be: grub2-install --force /dev/vda

You could try to launch it manually as indicated in the previous comment to see that you get the exact same error message. If you can reproduce, then the idea is to understand what command should be used to fix that problem.

[kevenw]

Added grub.cfg

Where is mount-me?: sh-4.2# mount-me sh: mount-me: command not found sh-4.2# find / -name mount-me sh-4.2#

[Bruno Cornec]

Replying to kevenw:

Where is mount-me?:

Sorry, I forgot it has been renamed as mr-mount-me

[kevenw]

I tried the suggestions and still saw the errors (see most recent attachment).

[Bruno Cornec]

Once you've done mr-mount-me, please bind mount /proc, /sys and /dev so the commands can find their devices and system files:

mount -o bind /proc /mnt/RESTORING/proc
mount -o bind /sys /mnt/RESTORING/sys
mount -o bind /dev /mnt/RESTORING/dev

Then you can go into the chroot and use it.

[kevenw]

That seems to have worked! (see attached)

Now what? :)

[Bruno Cornec]

What happens if you reboot your system after that ?

Normally that's the sequence that the mondorestore should launch, but your previous log was saying:

grub2-install: error: cannot find a GRUB drive for /dev/vda1.  Check your device.map.

So if this time it works, there is something which is different during the normal process compared to that manual sequence. I 'm not sure what as what I ask you to do, is what mondorestore should launch, but that could give me a place to look at.

One check could be to hack your mr-grub script and add a mount command in it to check whether these FS are indeed mounted at invocation time.

[kevenw]

Atfer the reboot, the VM started as expected! Yay!!!

I also tried the suggested mr-grub hack (see attached), and that worked also.

Can you add this into 3.3.0?

[Bruno Cornec]

I've made some changes to reinstall the boot loader. Could you try with the latest version under ​ftp://ftp.project-builder.org/test/rhel/7/x86_64/ to re-run an archive+restore test and let me know I I fixed your issue for good.

[kevenw]

Well, this is weird…

Somehow I missed the last update from you (I normally get e-mails, but don’t remember seeing one for this last update :( ). My apologies on that!

I tried your latest test version this afternoon, it failed again :( I can provide the logs, but are you sure the location you mentioned above is correct (it seems to still have the version from last August (your last message to test was in November))?