3.0.2-1 Fail to Restore from External Hard Disk

[SamK]

This ticket is raised at the request of Bruno as a result of a mailing list discussion. See archive: Date=June 10 2012 Title=3.0.2-1 Fail to Restore from External Hard Disk

A single mondorescue-1.iso backup is successfully created directly to an external hard disk and stored in /mondo.

Booting from the mondorescue.iso (mindi) CDROM

• Boot codes: interactive noresize
• Read from=Hard disk
• Prefix=mondorescue
• On what device do the ISO files live=/dev/sdb1
• What is disk format=ext4
• Are you restoring from an external hard drive=Yes
• At what path on this device can the ISO files be found=/mondo
• When mondorestore displays the message "Failed to find config/file archives. Choose another source?"
• ALT-F2
• mount "... /dev/sdb1 on /tmp/isodir type ext4 (ro,relatime,barrier=1,data=ordered)"
• ls -1 /mnt/isodir/mondo/*.iso "/tmp/isodir/mondo/mondorescue-1.iso /tmp/isodir/mondo/mondorescue.iso"

It seems that mondorestore mounts the external disk OK but does not use the path to the directory holding the backup ISO.

At the suggestion of Bruno mondorescue-1.iso was copied to the root of the external drive and the restore attempted again. This time the ISO is found but the restore fails at a later stage.

Booting from the mondorescue.iso (mindi) CDROM

• Boot codes: interactive noresize
• Read from=Hard disk
• Prefix=mondorescue
• On what device do the ISO files live=/dev/sdb1
• What is disk format=ext4
• Are you restoring from an external hard drive=Yes
• At what path on this device can the ISO files be found=/
• Editing Mountlist=OK
• Save mountlist and continue=Yes
• Erase and partition hard drives=Yes
• Restore all data=Yes
• A message is displayed "Please insert ISO #1 and press Enter"

At this point the mondorestore.log was saved as it is not possible to proceed further. The log is attached.

[bs27975]

Ditto, here.

First: Having followed FAQ 42 and written the mondoarchive.iso to a usb key, when booted, fdisk was not present.

• writing the iso out to a cdrom instead and booting from it, all was 'well.' (fdisk found, etc.)

So, the built usb did not build / boot correctly (syslinux booted but busybox didn't change into the loaded files)?

• curiously, booting from usb, it could not find the cdrom (which would be empty at the time).

When booting from the mondoarchive created mondorescue.iso:

• from what I could see, despite finding the .iso's on /dev/sdb6 [as evidenced by being listed at /tmp/isos (path?)], mondorescue ejected the (boot) cd, then asked for the first cd iso to be inserted.
• 'mount /path/to/mondorescue-1.iso /cdrom -o loop' got me beyond the request for the first iso, but then I went into an endless loop of it trying to build grub. I got the sense that it would drop me to a command prompt so I could manually repair things, but the screen merely blinked then took me to the next prompt. Since I had not fixed anything (no opportunity, no command line), I was in an endless loop [fix grub, (no) command line, fix grub, <repeat>], I had to reboot.
• post attempt 3 - I now understand this to have been the post-iso restore sequence. It did not read iso 2, and skipped to the post-iso restore stage.

Attempt number 2, specifying 'interactive noeject text':

• I got to a Yes / No prompt, and none of the following were accepted: Yes, Y, y, yes. This was somewhere around the mount list questions.
• rebooted, as nothing was being accepted, preventing me from proceeding.
• Again, post attempt 3, I now realize this to have been the post-iso restore process. However, the notext mode would not accept yes, and I bailed.

Attempt number 3, 'interactive noeject' with no reparitioning or formatting (as they were done in attempt 1), same situation (leading to this bug report - request for insertion of disc, when iso's are on an external usb disk)

• 'mount /path/to/mondorescue-1.iso /mnt/cdrom -o loop' (Alt-F2) used this time, per 'tail mondoarchive.log' (Difference from attempt 1: /mnt/cdrom instead of /cdrom. I believed at the time of attempt 1 that /cdrom was correct - I believe I saw a /cdrom directory, and so assumed that directory should be used for mounting, at that time. Point being, here: It appears being unable to find iso 1, it gave up and went on to post-iso restore activities.)
• repeat for each additional iso. /mnt/cdrom being automatically unmounted. i.e. no umount /mnt/cdrom necessary, it already having been done.
• restore completed successfully. (With 1 read error.)

[bs27975]

T628-BS mondo fdisk log.

[bs27975]

T628-BS mondo restore log - attempt 1 (failed)

[bs27975]

T628-BS mondo restore log - attempt 3 (kludge successful)

[Bruno Cornec]

From another user:

mondorestore *at the final stage* attempts to restore from a file called mondorescue-1.iso, although we have explicitly stated that the backup prefix is mondonetvis (so it should be looking for mondonetvis-1.iso, as it has correctly done at all previous stages of the process).

To prove the case, I changed the name of the backup file to mondorescue-1.iso and this time the restore process was successful. So there is a little bug there: mondorestore does not use the custom backup name at the final stage of the restore process, but the default one ("mondorescue").

[Bruno Cornec]

However he mentioned this doesn't seem to be related directly. However worth looking at.

[SamK]

Replying to bruno:

However he mentioned this doesn't seem to be related directly. However worth looking at.

Re-tested using names: mondorescue mondorescueverass-single The results are similar to those obtained in the original bug report, i.e. the problem is still present as reported.

[Bruno Cornec]

The revision [3158] should help improve the situation here, even if I have not tested it in a real case. A new test version for 3.0.4 is available under ​ftp://ftp.mondorescue.org/test so you could report. Please provide newest mondorestore.log log files as I've also added a bit more debuf in them for this case.

[SamK]

Replying to bruno:

The revision [3158] should help improve the situation here, even if I have not tested it in a real case. A new test version for 3.0.4 is available under ​ftp://ftp.mondorescue.org/test so you could report. Please provide newest mondorestore.log log files as I've also added a bit more debuf in them for this case.

Restoring interactively from mondorescue.iso (mindi) from CDROM still fails to find the external disk. The defaults it supplies for the prompted questions are incorrect. However these values seem to differ depending upon when the external disk is connected to the system booted by mindi.

Versions Tested:
mindi-busybox_1.18.5-3_i386.deb
mindi_2.1.620130628101458-0_i386.deb
mondo_3.0.420130624040329-0_i386.deb

Create backup via CLI
Command issued:
mondorescue -O -i -d /media/BACKUP-REPO/backup/home/nfs/mondo -G -9 -p mondoverass-single -s 700m -S /tmp -t /tmp -I "/" -E "|/media/BACKUP-REPO|"

Result:
success

Logs attached:
mindi.log.cli
mondoarchive.log.cli

TEST 1 Restore backup via GUI
External disk connected before power-on
Boot via mondorescue.iso (mindi) CDROM
Boot codes: interactive noresize
Responses=accept default values supplied by mondorestore (marked ++)

Restore=Interactively
Read from=Hard disk
Prefix=mondoverass-single ++
On what device do the ISO files live=/dev/sdb1 ++
What is disk format=ext4
Are you restoring from an external hard drive=Yes
At what path on this device can the ISO files be found=/media/BACKUP-REPO/backup/home/nfs/mondo ++
Error message="Failed to find config/file archives. Choose another source?"

Result:
fail

Log attached:
mondorestore.log.fail.pre-power

At first mondorestore menu (Welcome to Mondo)
ALT-F2
blkid -o list
Internal disk=/dev/sdb1
External disk=/dev/sda1
/dev/sdb1 mounted at /tmp/isodir
This is the exact opposite of the devices when mondoarchive created the backup.

Mondorestore supplies /dev/sdb1 as the device holding the backup. This was correct when mondoarchive was running. It is incorrect when mondorestore is running.

Mondorestore supplies the path to the ISO file=/media/BACKUP-REPO/backup/home/nfs/mondo. This was correct when mondorescue created the backup. It is incorrect when mondorestore is running.

TEST 2 Restore backup via GUI
External disk connected after boot up is complete (first mondodrestore menu)
Boot via mondorescue.iso (mindi) CDROM
Boot codes: interactive noresize
Responses=accept default values supplied by mondorestore (marked ++)

External disk connected immediately before entering the following response
Restore=Interactively
Read from=Hard disk
Prefix=mondoverass-single ++
On what device do the ISO files live=/dev/sdb1 ++
What is disk format=ext4
Are you restoring from an external hard drive=Yes
At what path on this device can the ISO files be found=/tmp/isodir ++
Error message="I have mounted the the device where the ISO files are stored"
Error message="I'm thinking... Cannot find config info on media"
Error message="Failed to find config/file archives. Choose another source?"

Result:
fail

Log attached:
mondorestore.log.fail.pre-responses

At first mondorestore menu (Welcome to Mondo)
ALT-F2
blkid -o list
Internal disk=/dev/sda1
External disk=/dev/sdb1
Both unmounted

At the first error message
ALT-F2
blkid -o list
/dev/sdb1 mounted at /tmp/isodir

So it mounts the correct device but does not search for and find the Prefix ISO. TEST 1 shows the path mondoarchive used to store the backup (/media/BACKUP-REPO/backup/home/nfs/mondo) is known and stored in the mindi ISO. Is it possible to get mondorestore to search that path as follows:

/tmp/isodir  (Success as it exists)
/tmp/isodir/media  (Fails as does not exist)
/tmp/isodir/BACKUP-REPO  (Fails as does not exist)
/tmp/isodir/backup  (Success as it exists in external disk)
/tmp/isodir/backup/home  (Success as it exists in external disk)
/tmp/isodir/backup/home/nfs  (Success as it exists in external disk)
/tmp/isodir/backup/home/nfs/mondo  (Success as it exists in external disk)
/tmp/isodir/backup/home/nfs/mondo/Prefix.iso  (Success as it exists in external disk)

SUMMARY
The device allocated to internal and external storage varies depending upon what is connected at boot up. This can result in an incorrect device allocation (TEST 1) or a correct device allocation (TEST 2).

TEST 2 mounts the correct device.

The backup prefix is correctly supplied in both tests.

An invalid path to ISO is supplied in both tests. A vaild path is contained within this invalid path. This is because the inital part of the invalid path includes directories that only exist on the internal disk of the system being backed-up. Those initial directories do not exist on the external disk. Once those initial directories are identified and removed the remaining path is valid as it does exist on the external disk.

Having cancelled the attempt to restore the backup, a message in received "Type 'exit' to reboot the PC". This fails to reboot. A further message is displayed "trap: usage trap [-lp] [[arg] signal_spec...]". It waits in this state forever.

[Bruno Cornec]

Replying to SamK:

Responses=accept default values supplied by mondorestore (marked ++)

Restore=Interactively
Read from=Hard disk
Prefix=mondoverass-single ++
On what device do the ISO files live=/dev/sdb1 ++
What is disk format=ext4
Are you restoring from an external hard drive=Yes
At what path on this device can the ISO files be found=/media/BACKUP-REPO/backup/home/nfs/mondo ++

I don't think this is correct. This is where it was at backup tie, but at least /media/BACKUP-REPO is probably not existing at that moment. So you should rather use backup/home/nfs/mondo IMO (not sure of that but you should be able to adapt.

At first mondorestore menu (Welcome to Mondo)
ALT-F2
blkid -o list
Internal disk=/dev/sdb1
External disk=/dev/sda1
/dev/sdb1 mounted at /tmp/isodir
This is the exact opposite of the devices when mondoarchive created the backup.

Well, this is probably linked to the order that is used to load driver; If you want something else, pleas euse forcemods at boot prompt to give your order.

TEST 2 Restore backup via GUI
External disk connected after boot up is complete (first mondodrestore menu)
Boot via mondorescue.iso (mindi) CDROM
Boot codes: interactive noresize
Responses=accept default values supplied by mondorestore (marked ++)

External disk connected immediately before entering the following response
Restore=Interactively
Read from=Hard disk
Prefix=mondoverass-single ++
On what device do the ISO files live=/dev/sdb1 ++
What is disk format=ext4
Are you restoring from an external hard drive=Yes
At what path on this device can the ISO files be found=/tmp/isodir ++

Again, this is not right. You should give the subdir on which the ISO is. mondorestore adds the /tmp/isodir itself to it.

So it mounts the correct device but does not search for and find the Prefix ISO. TEST 1 shows the path mondoarchive used to store the backup (/media/BACKUP-REPO/backup/home/nfs/mondo) is known and stored in the mindi ISO. Is it possible to get mondorestore to search that path as follows:

/tmp/isodir  (Success as it exists)
/tmp/isodir/media  (Fails as does not exist)
/tmp/isodir/BACKUP-REPO  (Fails as does not exist)
/tmp/isodir/backup  (Success as it exists in external disk)
/tmp/isodir/backup/home  (Success as it exists in external disk)
/tmp/isodir/backup/home/nfs  (Success as it exists in external disk)
/tmp/isodir/backup/home/nfs/mondo  (Success as it exists in external disk)
/tmp/isodir/backup/home/nfs/mondo/Prefix.iso  (Success as it exists in external disk)

Well we could, but asking the user to give the right information is probably the easiest way. Now maybe the messages are not clear enough, so feel free to provide better ones I could use.

[SamK]

Replying to bruno:

Well, this is probably linked to the order that is used to load driver; If you want something else, pleas euse forcemods at boot prompt to give your order.

Perhaps it was not as obvious as I had hoped in my report as it is when sitting in front of the screen. Maybe the following will help clarify what I meant.

Comparing the device order in Test1 v Test2

Mondorestore finds the internal and external disk in both tests. The device allocated is different depending upon whether the external disk is attached before power-on (Test1), or after boot-up has finished, but before supplying the responses to mondorestore (Test2).

Test2 assigns the device ids correctly, i.e. they correspond to the ids allocated when mondorescue created the backup. This success is what I was trying to indicate in the report. The Test2 method is the one that should be adopted as the standard method. I suggest that two steps are needed to make it apparent to users to follow this course.

Firstly, add the information to the published guidance.

Secondly, add a message to the very first screen displayed at boot-up (the one that asks for optional bootcodes). The message might be something like: "If restoring from a local external disk, do not connect it until the boot-up has finished."

ALTERNATIVE MESSAGES

"If restoring from a local external disk, do not connect it until prompted." Then at the 2nd menu (headed "Read from:"), when the user selects "Hard disk" add a screen that prompts for the external disk to be connected. Perhaps as message like: "Connect the external disk now. Wait 10 seconds for before proceeding to allow the system to find it"

My preference is strongly for the alternative pair of messages as they are much more obvious and therefore more user friendly.

Re: Automatically supplying the path to the ISO directory

Replying to SamK:

Is it possible to get mondorestore to search that path as follows:

[...]

Well we could, but asking the user to give the right information is probably the easiest way. Now maybe the messages are not clear enough, so feel free to provide better ones I could use.

I can see that it might be easier from a maintainers point of view, but less so from that of the end user. Running Mondorestore might happen very infrequently, perhaps many years after the back-up script was written and familiar. In this case the user may have forgotten the path used on the external disk. They will only become aware that the exact path is needed part way through the restore process. It will be an inconvenient discovery.

The following assumes the above method (Test2) has been used to correctly allocate the device ids.

Mondorestore currently supplies correct values to the prompts:

• Prefix
• Device on which ISO lives
• Format of the device (automatically detected when left blank)
• Are you restoring from an external hard drive (defaults to yes)

These are welcome as they are user friendly

Mondorestore currently supplies an incorrect (incomplete) value for the next prompt

• At what path on on this device can the ISO files be found

The value supplied (/tmp/isodir) is not needed and should be omitted. This will leave the field empty, which is preferable to the wrong value, but is not user friendly. As the correct path is already stored within the Mindi ISO, it seems a shame not to use it. This is particularly so becuase the user still retains the ability to input a path of their choice. It all goes to making mondorestore a more flexible, attractive user friendly tool.

For all the reasons given, would you be willing to consider again making use of the information that is already available within the Mindi ISO (or some other way of auto-populating the field with the correct information)?



Edit Supplementary feedback follows

Re: External disk device ID order
The above feedback (comment:10 in reply to: ↑ 9) is biased towards interactive restore. To accomodate also doing an unattended (nuke) restore it might be worth thinking about the following.

Boot screen message
"Restoring from a local external disk

• Interactively: do not connect the disk until the boot-up has finished
• Automatically: connect the disk now and specify it as a boot parameter e.g. sdXX nuke"

ALTERNATIVE MESSAGES

"Restoring from a local external disk

• Interactively: do not connect the disk until prompted (+++note to Bruno)
• Automatically: connect the disk now and specify it as a boot parameter e.g. sdXX nuke"

+++Issue the prompt at the 2nd menu (headed "Read from:"). When the user selects "Hard disk" add a screen that prompts for the external disk to be connected. Perhaps as message like: "Connect the external disk now. Wait 10 seconds for before proceeding to allow the system to find it"

My preference is strongly for the alternative pair of messages as they are much more obvious and therefore more user friendly.


Re: Automatically supplying the path to the ISO directory

I haven't tested the following.

If the bootcode "nuke" is used, does Mondrestore check that the backup ISO is found before "nuking" the disk? If so, it is another reason for automatically using the path to ISO that exists within the Mindi ISO.

[Bruno Cornec]

I have modifier the message at boot in order to improve stuff.

Wrt directory detection, and another message during restoration, it will have to wait till next version, sorry.

[SamK]

Replying to bruno:

I have modified the message at boot in rev [3173] in order to improve stuff.

Wrt directory detection, and another message during restoration, it will have to wait till next version, sorry.

That's OK, the next version is fine.

Some feedback re the new boot screen message, "If restoring from a local USB key, do not connect it until the boot-up has finished".

After testing I can confirm the partial success of this.

• It works OK when conducting an interactive restore.
• It prevents the use of the nuke boot parameter as there is no opportunity to connect the USB device.

Making mondorestore prompt the user to attach the USB storage device after the boot-up has finished would seem to be the answer. It will allow the nuke option to work in the usual and expected manner.