Changes between Version 2 and Version 3 of TroubleShooting


Ignore:
Timestamp:
Jan 25, 2007, 4:41:00 AM (12 years ago)
Author:
andree
Comment:

Moved from my page

Legend:

Unmodified
Added
Removed
Modified
  • TroubleShooting

    v2 v3  
    1 = Troubleshooting mindi =
     1= Trouble-Shooting mindi =
    22
    33Launch mindi using the verbose option of the shell:
     
    1010set -x
    1111}}}
     12
     13
     14= Trouble-Shooting mondo =
     15
     16mondo basically consists of two C programs, '''mondoarchive''' and '''mondorestore'''. To trouble-shoot mondo therefore may mean to debug. This sounds scarier than it is - just read on. ;-)
     17
     18== Creating Backtraces ==
     19Backtraces can be very helpful when trouble-shooting issues like segmentation faults. To create a useful backtrace, you need gdb (the GNU Debugger) installed and an application (and possibly libraries) with debugging symbols built in. The following will explain how to do this.
     20
     21=== gdb ===
     22
     23gdb should be part of your distribution just use your favourite way to install the package, e.g.
     24
     25{{{
     26apt-get install gdb
     27}}}
     28
     29for Debian and friends (such as Ubuntu) or
     30
     31{{{
     32yum install gdb
     33}}}
     34
     35for Fedora/RedHat/Mandriva
     36
     37
     38=== mondoarchive/mondorescue with debugging symbols ===
     39
     40To get '''mondoarchive''' and '''mondorescue''' with debugging symbols built in, you need to build from the source.
     41
     42Get the latest stable mondo source package from '''ftp://ftp.mondorescue.org/src/''', e.g. '''mondo-2.0.9.tar.gz''', unpack:
     43
     44{{{
     45tar xvzf mondo-2.0.9.tar.gz
     46}}}
     47
     48enter into the new directory and build using make:
     49
     50{{{
     51cd mondo-2.0.9
     52./configure --prefix=/usr
     53make
     54}}}
     55
     56You will end up with binary in the following locations which are non-stripped, i.e. they contain debugging symbols:
     57{{{
     58file mondo/mondoarchive/mondoarchive
     59mondo/mondoarchive/mondoarchive: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.0, dynamically linked (uses shared libs), not stripped
     60}}}
     61
     62and
     63
     64{{{
     65file mondo/mondorestore/mondorestore
     66mondo/mondorestore/mondorestore: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.0, dynamically linked (uses shared libs), not stripped
     67}}}
     68
     69Make backups of the original '''mondoarchive''' and '''mondorestore''' binaries and copy they newly created over the original ones.
     70
     71== Trouble-Shooting mondoarchive ==
     72
     73The best approach is to run the '''mondoarchive''' binary you just created with debugging symbols built in from its location in the built directory under gdb. So:
     74
     75{{{
     76cd mondo/mondoarchive
     77gdb ./mondoarchive
     78run <usual arguments you use>
     79}}}
     80
     81'''Note:''' Running it from within its build directory makes it so that more valuable information about source lines will be avilable in the backtrace.
     82
     83When the segmentation fault happens, enter:
     84
     85{{{
     86bt
     87}}}
     88
     89and send the output to the list.
     90
     91== Trouble-Shooting mondorestore ==
     92
     93If you do partial restore onto a live system, the same approach as described for '''mondoarchive''' can be used.
     94
     95However, more likely you will experience a segmentation fault during restore. To run a backtrace in that situation proceeed as follows:
     96
     97First, you need a '''mondorestore''' binary with debugging symbols. This should already been taken care of if you copied the newly compiled binaries as described above. Next, you need to make sure that gdb is available on your restore media. To achieve this, add this to '''/etc/mindi/deplist.txt''' before doing a mondoarchive run:
     98
     99{{{
     100gdb
     101libthread_db.so.1
     102}}}
     103
     104Boot the restore media into '''expert''' mode. Then start '''mondorestore''' like this:
     105
     106{{{
     107gdb /usr/sbin/mondorestore
     108run
     109}}}
     110
     111As described previously, once the segmentation fault happens, do:
     112
     113{{{
     114bt
     115}}}
     116
     117and send the output to the list. (If you can't get the backtrace copied as text, you can use a photo of the screen as the last resort.
     118
     119== Advanced Topics ==
     120
     121=== Attaching to Running Processes ===
     122
     123You can attach to a running process using:
     124
     125{{{
     126gdb /usr/sbin/mondorestore <pid>
     127}}}
     128
     129where <pid> is the process ID.
     130
     131This can be particulary useful when running '''mondoarchive''' with the '-g' or when running '''mondorestore'''.
     132
     133=== Using libraries with debugging symbols ===
     134
     135The libraries used by a binary can be determined using the ldd command, e.g.:
     136
     137{{{
     138ldd /usr/sbin/mondoarchive
     139                libmondo.so.2 => /usr/lib/libmondo.so.2 (0xb7f8d000)
     140        libmondo-newt.so.1 => /usr/lib/libmondo-newt.so.1 (0xb7f82000)
     141        libnewt.so.0.51 => /usr/lib/libnewt.so.0.51 (0xb7f71000)
     142        libdl.so.2 => /lib/tls/libdl.so.2 (0xb7f6e000)
     143        libpthread.so.0 => /lib/tls/libpthread.so.0 (0xb7f5f000)
     144        libc.so.6 => /lib/tls/libc.so.6 (0xb7e2a000)
     145        libslang.so.1-UTF8 => /lib/libslang.so.1-UTF8 (0xb7db7000)
     146        libm.so.6 => /lib/tls/libm.so.6 (0xb7d94000)
     147        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0xb7fea000)
     148}}}
     149
     150some of those libraries may come with debugging symbols built in as an alternative package, others can be buiolt from scratch and installed with debugging sysmbols installed. Using your distribtuion's standard built process is probably a good idea for this.
     151
     152=== Worthwhile gcc Flags ===
     153-Wextra -Wshadow -Wstack-protector -fstack-protector
     154
     155= Getting the entire kernel log on restore media =
     156
     157The kernel ring buffer that '''dmesg''' reads defaults to 32k on recent kernels. This is not enough to capture the entire sequence of kernel message when Mondo Rescue boots off a restore media.
     158
     159To increase the kernel ring buffer to 128k at boot time (and without recompilation) add the following kernel boot parameter:
     160{{{
     161log_buf_len=128k
     162}}}
     163e.g.
     164{{{
     165expert log_buf_len=128k
     166}}}
     167'''dmesg''' needs to be told what buffer size to use to ensure that everything is displayed from the start. The '''-s''' parameter can be used for this like this:
     168{{{
     169dmesg -s 131072 | less
     170}}}