source: MondoRescue/branches/stable/mondo/src/common/libmondo-filelist.c @ 1770

Last change on this file since 1770 was 1770, checked in by Bruno Cornec, 12 years ago
  • Better output for mindi-busybox revision
  • Remove dummy file created on NFS - report from Arnaud Tiger <arnaud.tiger_at_hp.com>
  • strace useful for debug
  • fix new versions for pb (2.0.0 for mindi and 1.7.2 for mindi-busybox)
  • fix build process for mindi-busybox + options used in that version (dd for label-partitions-as-necessary)
  • fix typo in label-partitions-as-necessary which doesn't seem to work
  • Update to busybox 1.7.2
  • perl is now required at restore time to support uuid swap partitions (and will be used for many other thigs

in the future for sure)

  • next mindi version will be 2.0.0 due to all the changes made in it (udev may break working distros)
  • small optimization in mindi on keyboard handling (one single find instead of multiple)
  • better interaction for USB device when launching mindi manually
  • attempt to automatically guess block disk size for ramdisk
  • fix typos in bkphw
  • Fix the remaining problem with UUID support for swap partitions
  • Updates mondoarchive man page for USB support
  • Adds preliminary Hardware support to mindi (Proliant SSSTK)
  • Tries to add udev support also for rhel4
  • Fix UUID support which was still broken.
  • Be conservative in test for the start-nfs script
  • Update config file for mindi-busybox for 1.7.2 migration
  • Try to run around a busybox bug (1.2.2 pb on inexistant links)
  • Add build content for mindi-busybox in pb
  • Remove distributions content for mindi-busybox
  • Fix a warning on inexistant raidtab
  • Solve problem on tmpfs in restore init (Problem of inexistant symlink and busybox)
  • Create MONDO_CACHE and use it everywhere + creation at start
  • Really never try to eject a USB device
  • Fix a issue with &> usage (replaced with 1> and 2>)
  • Adds magic file to depllist in order to have file working + ldd which helps for debugging issues
  • tty modes correct to avoid sh error messages
  • Use ext3 normally and not ext2 instead
  • USB device should be corrected after reading (take 1st part)
  • Adds a mount_USB_here function derived from mount_CDROM_here
  • usb detection place before /dev detection in device name at restore time
  • Fix when restoring from USB: media is asked in interactive mode
  • Adds USB support for mondorestore
  • mount_cdrom => mount_media
  • elilo.efi is now searched throughout /boot/efi and not in a fixed place as there is no standard
  • untar-and-softlink => untar (+ interface change)
  • suppress useless softlinks creation/removal in boot process
  • avoids udevd messages on groups
  • Increase # of disks to 99 as in mindi at restore time (should be a conf file parameter)
  • skip existing big file creation
  • seems to work correctly for USB mindi boot
  • Adds group and tty link to udev conf
  • Always load usb-torage (even 2.6) to initiate USB bus discovery
  • Better printing of messages
  • Attempt to fix a bug in supporting OpenSusE 10.3 kernel for initramfs (mindi may now use multiple regex for kernel initrd detection)
  • Links were not correctly done as non relative for modules in mindi
  • exclusion of modules denied now works
  • Also create modules in their ordinary place, so that classical modprobe works + copy modules.dep
  • Fix bugs for DENY_MODS handling
  • Add device /dev/console for udev
  • ide-generic should now really be excluded
  • Fix a bug in major number for tty
  • If udev then adds modprobe/insmod to rootfs
  • tty0 is also cretaed with udev
  • ide-generic put rather in DENY_MODS
  • udevd remove from deplist s handled in mindi directly
  • better default for mindi when using --usb
  • Handles dynamically linked busybox (in case we want to use it soon ;-)
  • Adds fixed devices to create for udev
  • ide-generic should not be part of the initrd when using libata v2
  • support a dynamically linked udev (case on Ubuntu 7.10 and Mandriva 2008.0 so should be quite generic) This will give incitation to move to dyn. linked binaries in the initrd which will help for other tasks (ia6 4)
  • Improvement in udev support (do not use cl options not available in busybox)
  • Udev in mindi
    • auto creation of the right links at boot time with udev-links.conf(from Mandriva 2008.0)
    • rework startup of udev as current makes kernel crash (from Mandriva 2008.0)
    • add support for 64 bits udev
  • Try to render MyInsmod? silent at boot time
  • Adds udev support (mandatory for newest distributions to avoid remapping of devices in a different way as on the original system)
  • We also need vaft format support for USB boot
  • Adds libusual support (Ubuntu 7.10 needs it for USB)
  • Improve Ubuntu/Debian? keyboard detection and support
  • pbinit adapted to new pb (0.8.10). Filtering of docs done in it
  • Suppress some mondo warnings and errors on USB again
  • Tries to fix lack of files in deb mindi package
  • Verify should now work for USB devices
  • More log/mesages improvement for USB support
  • - Supress g_erase_tmpdir_and_scratchdir
  • Improve some log messages for USB support
  • Try to improve install in mindi to avoid issues with isolinux.cfg not installed vene if in the pkg :-(
  • Improve mindi-busybox build
  • In conformity with pb 0.8.9
  • Add support for Ubuntu 7.10 in build process
  • Add USB Key button to Menu UI (CD streamer removed)
  • Attempt to fix error messages on tmp/scratch files at the end by removing those dir at the latest possible.
  • Fix a bug linked to the size of the -E param which could be used (Arnaud Tiger/René? Ribaud).
  • Integrate ~/.pbrc content into mondorescue.pb (required project-builder >= 0.8.7)
  • Put mondorescue in conformity with new pb filtering rules
  • Add USB support at restore time (no test done yet). New start-usb script PB varibale added where useful
  • Unmounting USB device before removal of temporary scratchdir
  • Stil refining USB copy back to mondo (one command was not executed)
  • No need to have the image subdor in the csratchdir when USB.
  • umount the USB partition before attempting to use it
  • Remove useless copy from mindi to mondo at end of USB handling

(risky merge, we are raising the limits of 2 diverging branches. The status of stable is not completely sure as such. Will need lots of tests, but it's not yet done :-()
(merge -r1692:1769 $SVN_M/branches/2.2.5)

  • Property svn:keywords set to Id
File size: 48.2 KB
Line 
1/* libmondo-filelist.c
2   $Id: libmondo-filelist.c 1770 2007-11-06 10:01:53Z bruno $
3*/
4
5/**
6 * @file
7 * Functions which create, chop, and edit the filelist.
8 */
9
10#include "my-stuff.h"
11#include "mondostructures.h"
12#include "libmondo-filelist.h"
13#include "libmondo-string-EXT.h"
14#include "libmondo-files-EXT.h"
15#include "libmondo-fork-EXT.h"
16#include "newt-specific-EXT.h"
17#include "libmondo-tools-EXT.h"
18
19#include "mr_str.h"
20#include "mr_mem.h"
21#include "mr_msg.h"
22#include "mr_gettext.h"
23
24#include <time.h>
25#include <stdio.h>
26#include <sys/types.h>
27#include <sys/stat.h>
28#include <dirent.h>
29#include <errno.h>
30#include <stdio.h>
31
32
33/**
34 * The maximum depth of directories to put in the skeleton filelist.
35 * This is a balance between performance and a good progress indicator.
36 */
37#define MAX_SKEL_DEPTH 3
38
39
40extern char *MONDO_LOGFILE;
41
42/* Reference to global bkpinfo */
43extern struct s_bkpinfo *bkpinfo;
44
45int mondo_makefilelist(char *logfile, char *tmpdir, char *scratchdir,
46                       char *include_paths, char *excp, bool differential,
47                       char *userdef_filelist);
48
49
50/*@unused@*/
51//static char cvsid[] = "$Id: libmondo-filelist.c 1770 2007-11-06 10:01:53Z bruno $";
52
53/**
54 * Number of lines in the filelist last loaded.
55 * @warning This implies that two filesets cannot be loaded at once.
56 * @ingroup globalGroup
57 */
58long g_original_noof_lines_in_filelist = 0;
59
60/**
61 * Number of filesets in the current backup.
62 * @ingroup globalGroup
63 */
64long g_noof_sets = 0;
65
66extern bool g_text_mode;
67extern newtComponent g_progressForm;
68extern int g_currentY;
69extern int g_noof_rows;
70
71extern char *g_getfacl;
72extern char *g_getfattr;
73
74/**
75 * @addtogroup filelistGroup
76 * @{
77 */
78/**
79 * Call chop_filelist() to chop the filelist into sets.
80 * @param bkpinfo The backup information structure. Fields used:
81 * - @c bkpinfo->image_devs
82 * - @c bkpinfo->optimal_set_size
83 * - @c bkpinfo->scratchdir
84 * - @c bkpinfo->tmpdir
85 * @see chop_filelist
86 */
87int call_filelist_chopper()
88{
89    /*@ buffers *********************** */
90    char *dev;
91    char *filelist;
92    char *tempfile;
93    long noof_sets;
94
95    /*@ pointers ********************** */
96    char *ptr;
97    FILE *fout;
98
99    /*@ int *************************** */
100    int i, retval = 0;
101
102    mvaddstr_and_log_it(g_currentY, 0, _("Dividing filelist into sets"));
103
104    log_to_screen(_("Dividing filelist into sets. Please wait."));
105    i = 0;
106    mr_asprintf(&filelist, "%s/archives/filelist.full", bkpinfo->scratchdir);
107    if (!does_file_exist(filelist)) {
108        log_it("filelist %s not found", filelist);
109        fatal_error("call_filelist_chopper() -- filelist not found!");
110    }
111
112    noof_sets =
113        chop_filelist(filelist, bkpinfo->tmpdir,
114                      bkpinfo->optimal_set_size);
115    mr_free(filelist);
116
117    estimate_noof_media_required(noof_sets);    // for cosmetic purposes
118
119    mr_asprintf(&tempfile, "%s/biggielist.txt", bkpinfo->tmpdir);
120    if (!(fout = fopen(tempfile, "a"))) {
121        log_OS_error("Cannot append to biggielist");
122        retval++;
123    }
124    mr_free(tempfile);
125    log_it(bkpinfo->image_devs);
126
127    ptr = bkpinfo->image_devs;
128
129    while (ptr && *ptr) {
130        mr_asprintf(&dev, ptr);
131        log_it("Examining imagedev %s", dev);
132        for (i = 0; i < (int) strlen(dev) && dev[i] != ' '; i++);
133        dev[i] = '\0';
134        if (!strlen(dev)) {
135            continue;
136        }
137        fprintf(fout, "%s\n", dev);
138        log_it("Adding '%s' to biggielist", dev);
139        mr_free(dev);
140
141        if ((ptr = strchr(ptr, ' '))) {
142            ptr++;
143        }
144    }
145    paranoid_fclose(fout);
146    mvaddstr_and_log_it(g_currentY++, 74, "Done.");
147
148    return (retval);
149}
150
151
152
153int sort_file(char *orig_fname)
154{
155    char *tmp_fname;
156    char *command;
157    int retval = 0;
158
159    mr_msg(5, "Sorting file %s", orig_fname);
160    mr_asprintf(&tmp_fname, "%s/sortfile", bkpinfo->tmpdir);
161
162    if (!does_file_exist(orig_fname)) {
163        mr_msg(2, "file %s empty", orig_fname);
164        return (0);
165    }                           // no sense in trying to sort an empty file
166
167    mr_asprintf(&command, "sort %s > %s 2>> %s", orig_fname, tmp_fname,
168             MONDO_LOGFILE);
169    retval = system(command);
170    mr_free(command);
171
172    if (retval) {
173        mr_msg(2, "Failed to sort %s - oh dear", orig_fname);
174    } else {
175        mr_msg(5, "Sorted %s --> %s OK. Copying it back to %s now",
176                orig_fname, tmp_fname, orig_fname);
177        mr_asprintf(&command, "mv -f %s %s", tmp_fname, orig_fname);
178        retval += run_program_and_log_output(command, 5);
179        mr_free(command);
180
181        if (retval) {
182            mr_msg(2, "Failed to copy %s back to %s - oh dear", tmp_fname,
183                    orig_fname);
184        } else {
185            mr_msg(5, "%s was sorted OK.", orig_fname);
186        }
187    }
188    mr_free(tmp_fname);
189    mr_msg(5, "Finished sorting file %s", orig_fname);
190    return (retval);
191}
192
193
194/**
195 * Chop the filelist into sets.
196 * Each fileset is a list of files whose total (uncompressed) size is usually
197 * about X KB. Files bigger than 8X KB are placed in a "biggielist"; they will
198 * be sliced and compressed separately from the regular files.
199 *
200 * @param filelist The big filelist (filelist.full) to chop up.
201 * @param outdir The directory to place the files (filelist.N where N is
202 * an integer, biggielist.txt, and LAST-FILELIST-NUMBER) created
203 * @param maxsetsizeK Optimal size of a fileset (X above).
204 * @return number of errors encountered (0 for success).
205 */
206int chop_filelist(char *filelist, char *outdir, long maxsetsizeK)
207{
208/*@ long ****************************************/
209    long lino = 0;
210    long max_sane_size_for_a_file;
211    long curr_set_size;
212    long noof_lines;
213    long siz;
214
215    /*@ int **************************************** */
216    int i;
217    size_t n = 0;
218    long curr_set_no;
219
220    /*@ buffers ************************************* */
221    char *outfname;
222    char *biggie_fname;
223    char *incoming = NULL;
224    char *tmp;
225
226    /*@ pointers *********************************** */
227    FILE *fin;
228    FILE *fout;
229    FILE *fbig;
230
231    /*@ structures ********************************* */
232    struct stat buf;
233
234    assert_string_is_neither_NULL_nor_zerolength(filelist);
235    assert_string_is_neither_NULL_nor_zerolength(outdir);
236    assert(maxsetsizeK > 0);
237
238    max_sane_size_for_a_file = 64L * 1024L;
239// max_sane_size_for_a_file = maxsetsizeK*2;
240//  if (max_sane_size_for_a_file > 32*1024)
241//    { max_sane_size_for_a_file = 32*1024; }
242
243    log_it("filelist=%s;", filelist);
244    open_evalcall_form("Dividing filelist into sets");
245    noof_lines = count_lines_in_file(filelist);
246    if (!(fin = fopen(filelist, "r"))) {
247        log_OS_error("Cannot openin filelist");
248        return (0);
249    }
250    curr_set_no = 0;
251    curr_set_size = 0;
252    mr_asprintf(&outfname, "%s/filelist.%ld", outdir, curr_set_no);
253    mr_asprintf(&biggie_fname, "%s/biggielist.txt", outdir);
254    log_it("outfname=%s; biggie_fname=%s", outfname, biggie_fname);
255    if (!(fbig = fopen(biggie_fname, "w"))) {
256        log_OS_error("Cannot openout biggie_fname");
257        return (0);
258    }
259    if (!(fout = fopen(outfname, "w"))) {
260        log_OS_error("Cannot openout outfname");
261        return (0);
262    }
263    mr_getline(&incoming, &n, fin);
264    while (!feof(fin)) {
265        lino++;
266        i = strlen(incoming) - 1;
267        if (i < 0) {
268            i = 0;
269        }
270        if (incoming[i] < 32) {
271            incoming[i] = '\0';
272        }
273        if (!strncmp(incoming, "/dev/", 5)) {
274            siz = 1;
275        } else if (lstat(incoming, &buf) != 0) {
276            siz = 0;
277        } else {
278            siz = (long) (buf.st_size >> 10);
279        }
280        if (siz > max_sane_size_for_a_file)
281// && strcmp(incoming+strlen(incoming)-4, ".bz2") && strcmp(incoming+strlen(incoming)-4, ".tbz"))
282        {
283            fprintf(fbig, "%s\n", incoming);
284        } else {
285            curr_set_size += siz;
286            fprintf(fout, "%s\n", incoming);
287            if (curr_set_size > maxsetsizeK) {
288                paranoid_fclose(fout);
289                sort_file(outfname);
290                mr_free(outfname);
291
292                curr_set_no++;
293                curr_set_size = 0;
294                mr_asprintf(&outfname, "%s/filelist.%ld", outdir,
295                         curr_set_no);
296                if (!(fout = fopen(outfname, "w"))) {
297                    log_OS_error("Unable to openout outfname");
298                    return (0);
299                }
300                if (noof_lines != 0) {
301                    update_evalcall_form((int) (lino * 100 / noof_lines));
302                }
303            }
304        }
305        mr_getline(&incoming, &n, fin);
306    }
307    paranoid_fclose(fin);
308    paranoid_fclose(fout);
309    paranoid_fclose(fbig);
310
311    if (length_of_file(outfname) <= 2) {
312        unlink(outfname);
313        g_noof_sets--;
314    }
315    g_noof_sets = curr_set_no;
316    sort_file(outfname);
317    sort_file(biggie_fname);
318    mr_free(biggie_fname);
319    mr_free(outfname);
320
321    mr_asprintf(&outfname, "%s/LAST-FILELIST-NUMBER", outdir);
322    mr_asprintf(&tmp, "%ld", curr_set_no);
323    if (write_one_liner_data_file(outfname, tmp)) {
324        log_OS_error
325            ("Unable to echo write one-liner to LAST-FILELIST-NUMBER");
326        return (0);
327    }
328    mr_free(outfname);
329    mr_free(tmp);
330
331    if (curr_set_no == 0) {
332        mr_asprintf(&tmp, "Only one fileset. Fine.");
333    } else {
334        mr_asprintf(&tmp, "Filelist divided into %ld sets", curr_set_no + 1);
335    }
336    mr_msg(1, tmp);
337    mr_free(tmp);
338    close_evalcall_form();
339    /* This is to work around an obscure bug in Newt; open a form, close it,
340       carry on... I don't know why it works but it works. If you don't do this
341       then update_progress_form() won't show the "time taken / time remaining"
342       line. The bug only crops up AFTER the call to chop_filelist(). Weird. */
343    if (!g_text_mode) {
344        open_progress_form("", "", "", "", 100);
345        newtPopHelpLine();
346        newtFormDestroy(g_progressForm);
347        newtPopWindow();
348    }
349    return (curr_set_no + 1);
350}
351
352
353/**
354 * Free all the memory used by a filelist structure.
355 * Since this may take a long time for large filelists, a progress bar will be displayed.
356 * @param filelist The filelist to free.
357 */
358void free_filelist(struct s_node *filelist)
359{
360    /*@ int's ******************************************************* */
361    static int depth = 0;
362    int percentage;
363
364    /*@ long's ****************************************************** */
365    static long i = 0;
366
367    /*@ end vars **************************************************** */
368
369    assert(filelist != NULL);
370    if (depth == 0) {
371        open_evalcall_form("Freeing memory");
372        log_to_screen(_("Freeing memory formerly occupied by filelist"));
373    }
374    depth++;
375
376    if (filelist->ch == '\0') {
377        if (!(i++ % 1111) && (g_original_noof_lines_in_filelist != 0)) {
378            percentage =
379                (int) (i * 100 / g_original_noof_lines_in_filelist);
380            update_evalcall_form(percentage);
381
382        }
383    }
384
385    if (filelist->right) {
386        free_filelist(filelist->right);
387        filelist->right = NULL;
388    }
389    if (filelist->down) {
390/*      if (!(i++ %39999)) { update_evalcall_form(0); } */
391        free_filelist(filelist->down);
392        filelist->down = NULL;
393    }
394    filelist->ch = '\0';
395    mr_free(filelist);
396    depth--;
397    if (depth == 0) {
398        close_evalcall_form();
399        log_it("Finished freeing memory");
400    }
401}
402
403
404int call_exe_and_pipe_output_to_fd(char *sys_call, FILE * pout)
405{
406    FILE *pattr;
407    char *tmp = NULL;
408    size_t n = 0;
409
410    pattr = popen(sys_call, "r");
411    if (!pattr) {
412        mr_msg(1, "Failed to open fattr() %s", sys_call);
413        return (1);
414    }
415    if (feof(pattr)) {
416        mr_msg(1, "Failed to call fattr() %s", sys_call);
417        paranoid_pclose(pattr);
418        return (2);
419    }
420    for (mr_getline(&tmp, &n, pattr); !feof(pattr); mr_getline(&tmp, &n, pattr)) {
421        fputs(tmp, pout);
422    }
423    paranoid_pclose(pattr);
424    mr_free(tmp);
425    return (0);
426}
427
428
429int gen_aux_list(char *filelist, char *syscall_sprintf,
430                 char *auxlist_fname)
431{
432    FILE *fin;
433    FILE *pout;
434    char *pout_command;
435    char *sys_call;
436    char *file_to_analyze = NULL;
437    char *strtmp = NULL;
438    char *tmp = NULL;
439    int i;
440    size_t n = 0;
441
442    if (!(fin = fopen(filelist, "r"))) {
443        mr_msg(1, "Cannot openin filelist %s", filelist);
444        return (1);
445    }
446    mr_asprintf(&pout_command, "gzip -c1 > %s", auxlist_fname);
447    if (!(pout = popen(pout_command, "w"))) {
448        mr_msg(1, "Cannot openout auxlist_fname %s", auxlist_fname);
449        fclose(fin);
450        mr_free(pout_command);
451        return (4);
452    }
453    mr_free(pout_command);
454
455    for (mr_getline(&file_to_analyze, &n, fin); !feof(fin);
456         mr_getline(&file_to_analyze, &n, fin)) {
457        i = strlen(file_to_analyze);
458        if (i > 0 && file_to_analyze[i - 1] < 32) {
459            file_to_analyze[i - 1] = '\0';
460        }
461        mr_msg(8, "Analyzing %s", file_to_analyze);
462        /* BERLIOS : to be checked */
463        tmp = mr_stresc(file_to_analyze, "`$\\\"", BACKSLASH);
464        mr_asprintf(&strtmp, syscall_sprintf, tmp);
465        mr_free(tmp);
466        mr_asprintf(&sys_call, "%s 2>> /dev/null", strtmp);
467        mr_free(strtmp);
468        call_exe_and_pipe_output_to_fd(sys_call, pout);
469        mr_free(sys_call);
470    }
471    paranoid_fclose(fin);
472    paranoid_pclose(pout);
473    mr_free(file_to_analyze);
474    return (0);
475}
476
477
478int get_acl_list(char *filelist, char *facl_fname)
479{
480    char *command = NULL;
481    int retval = 0;
482
483    if (g_getfacl != NULL) {
484        mr_asprintf(&command, "touch %s", facl_fname);
485        run_program_and_log_output(command, 8);
486        mr_free(command);
487
488        mr_asprintf(&command,
489                 "getfacl --all-effective -P %s 2>> %s | gzip -c1 > %s 2>> %s",
490                 filelist, MONDO_LOGFILE, facl_fname, MONDO_LOGFILE);
491        iamhere(command);
492        retval = system(command);
493        mr_free(command);
494    }
495    return (retval);
496}
497
498
499int get_fattr_list(char *filelist, char *fattr_fname)
500{
501    char *command = NULL;
502    int retval = 0;
503
504    if (g_getfattr != NULL) {
505        mr_asprintf(&command, "touch %s", fattr_fname);
506        run_program_and_log_output(command, 8);
507        mr_free(command);
508
509        retval =
510            gen_aux_list(filelist, "getfattr --en=hex -P -d \"%s\"",
511                         fattr_fname);
512    }
513    return (retval);
514}
515
516
517int set_EXAT_list(char *orig_msklist, char *original_exat_fname,
518                  char *executable)
519{
520    const int my_depth = 8;
521    char *command, *syscall_pin, *syscall_pout;
522    char *incoming = NULL;
523    char *current_subset_file = NULL;
524    char *current_master_file, *masklist;
525    int retval = 0;
526    int i;
527    size_t n = 0;
528    char *p, *q;
529    FILE *pin, *pout, *faclin;
530
531    mr_msg(1, "set_EXAT_list(%s, %s, %s)", orig_msklist,
532            original_exat_fname, executable);
533    if (!orig_msklist || !orig_msklist[0]
534        || !does_file_exist(orig_msklist)) {
535        mr_msg(1,
536                "No masklist provided. I shall therefore set ALL attributes.");
537        mr_asprintf(&command, "gzip -dc %s | %s --restore - 2>> %s",
538                 original_exat_fname, executable, MONDO_LOGFILE);
539        mr_msg(1, "command = %s", command);
540        retval = system(command);
541        mr_free(command);
542        mr_msg(1, "Returning w/ retval=%d", retval);
543        return (retval);
544    }
545    if (length_of_file(original_exat_fname) <= 0) {
546        mr_msg(1,
547                "original_exat_fname %s is empty or missing, so no need to set EXAT list",
548                original_exat_fname);
549        return (0);
550    }
551    mr_asprintf(&masklist, "%s/masklist", bkpinfo->tmpdir);
552    mr_asprintf(&command, "cp -f %s %s", orig_msklist, masklist);
553    run_program_and_log_output(command, 1);
554    mr_free(command);
555
556    sort_file(masklist);
557    mr_asprintf(&syscall_pin, "gzip -dc %s", original_exat_fname);
558    mr_asprintf(&syscall_pout, "%s --restore - 2>> %s", executable,
559             MONDO_LOGFILE);
560
561    mr_msg(1, "syscall_pin = %s", syscall_pin);
562    mr_msg(1, "syscall_pout = %s", syscall_pout);
563    pout = popen(syscall_pout, "w");
564    if (!pout) {
565        iamhere("Unable to openout to syscall_pout");
566        return (1);
567    }
568    mr_free(syscall_pout);
569
570    pin = popen(syscall_pin, "r");
571    if (!pin) {
572        pclose(pout);
573        iamhere("Unable to openin from syscall");
574        return (1);
575    }
576    mr_free(syscall_pin);
577
578    faclin = fopen(masklist, "r");
579    if (!faclin) {
580        pclose(pin);
581        pclose(pout);
582        iamhere("Unable to openin masklist");
583        return (1);
584    }
585//  printf("Hi there. Starting the loop\n");
586
587    mr_getline(&current_subset_file, &n, faclin);
588    mr_getline(&incoming, &n, pin);
589    while (!feof(pin) && !feof(faclin)) {
590//      printf("incoming = %s", incoming);
591
592        mr_asprintf(&current_master_file, incoming + 8);
593
594        p = current_subset_file;
595        if (*p == '/') {
596            p++;
597        }
598        i = strlen(p);
599        if (i > 0 && p[i - 1] < 32) {
600            p[i - 1] = '\0';
601        }
602
603
604        q = current_master_file;
605        if (*q == '/') {
606            q++;
607        }
608        i = strlen(q);
609        if (i > 0 && q[i - 1] < 32) {
610            q[i - 1] = '\0';
611        }
612
613        i = strcmp(p, q);
614        mr_msg(my_depth, "'%s' v '%s' --> %d\n", p, q, i);
615
616//      printf("%s v %s --> %d\n", p, q, i);
617
618        if (i < 0) {            // read another subset file in.
619            mr_msg(my_depth, "Reading next subset line in\n\n");
620            mr_getline(&current_subset_file, &n, faclin);
621            continue;
622        }
623
624        if (!i) {
625            fputs(incoming, pout);
626        }
627        mr_getline(&incoming, &n, pin);
628        if (!i) {
629            mr_msg(my_depth, "Copying master %s", q);
630        }
631//      if (!i) { printf("Match --- %s\n", q); }
632
633        while (!feof(pin) && strncmp(incoming, "# file: ", 8)) {
634            if (!i) {
635
636//    printf("%s", incoming);
637
638                fputs(incoming, pout);
639            }
640            mr_getline(&incoming, &n, pin);
641        }
642        if (!i) {
643            mr_getline(&current_subset_file, &n, faclin);
644        }
645        mr_free(current_master_file);
646    }
647    mr_free(current_subset_file);
648
649    while (!feof(pin)) {
650        mr_getline(&incoming, &n, pin);
651    }
652    mr_free(incoming);
653    fclose(faclin);
654    pclose(pin);
655    pclose(pout);
656
657//  printf("OK, loop is done\n");
658
659    unlink(masklist);
660    mr_free(masklist);
661
662    return (retval);
663}
664
665
666int set_fattr_list(char *masklist, char *fattr_fname)
667{
668    if (find_home_of_exe("setfattr")) {
669        return (set_EXAT_list(masklist, fattr_fname, "setfattr"));
670    } else {
671        mr_msg(1, "ERROR: set_EXAT_list: setfattr doesn't exist");
672        return(0);
673    }
674}
675
676
677
678int set_acl_list(char *masklist, char *acl_fname)
679{
680    if (find_home_of_exe("setfacl")) {
681        return (set_EXAT_list(masklist, acl_fname, "setfacl"));
682    } else {
683        mr_msg(1, "ERROR: set_EXAT_list: setfacl doesn't exist");
684        return(0);
685    }
686}
687
688
689/**
690 * Get the number of the last fileset in the backup.
691 * @param bkpinfo The backup information structure. Only the @c bkpinfo->tmpdir field is used.
692 * @return The last filelist number.
693 * @note This function should only be called at restore-time.
694 */
695int get_last_filelist_number()
696{
697    /*@ buffers ***************************************************** */
698    char *val_sz = NULL;
699    char *cfg_fname = NULL;
700
701    /*@ long ******************************************************** */
702    int val_i = 0;
703
704    /*@ end vars **************************************************** */
705
706    assert(bkpinfo != NULL);
707
708    malloc_string(val_sz);
709    strcpy(val_sz,"");
710    mr_asprintf(&cfg_fname, "%s/mondo-restore.cfg", bkpinfo->tmpdir);
711    read_cfg_var(cfg_fname, "last-filelist-number", val_sz);
712    val_i = atoi(val_sz);
713    if (val_i <= 0) {
714        val_i = 500;
715    }
716    mr_free(cfg_fname);
717    mr_free(val_sz);
718    return (val_i);
719}
720
721
722/**
723 * Add a string at @p startnode.
724 * @param startnode The node to start at when searching for where to add the string.
725 * @param string_to_add The string to add.
726 * @return 0 for success, 1 for failure.
727 * @bug I don't understand this function. Would someone care to explain it?
728 */
729
730int add_string_at_node(struct s_node *startnode, char *string_to_add)
731{
732
733    int noof_chars;
734    int i;
735    int res;
736
737    struct s_node *node, *newnode;
738
739    char char_to_add;
740
741    const bool sosodef = FALSE;
742
743    static int depth = 0;
744    static char original_string[MAX_STR_LEN];
745
746    assert(startnode != NULL);
747    assert(string_to_add != NULL);
748
749    if (!depth) {
750        strcpy(original_string, string_to_add);
751    }
752
753    noof_chars = strlen(string_to_add) + 1; // we include the '\0'
754
755    // walk across tree if necessary
756    node = startnode;
757    char_to_add = string_to_add[0];
758    if (node->right != NULL && node->ch < char_to_add) {
759        mr_msg(7, "depth=%d --- going RIGHT ... %c-->%c", depth,
760                char_to_add, node->ch, (node->right)->ch);
761        return (add_string_at_node(node->right, string_to_add));
762    }
763
764    // walk down tree if appropriate
765    if (node->down != NULL && node->ch == char_to_add) {
766        mr_msg(7, "depth=%d char=%c --- going DOWN", depth, char_to_add);
767        depth++;
768        res = add_string_at_node(node->down, string_to_add + 1);
769        depth--;
770        return (res);
771    }
772
773    if (char_to_add == '\0' && node->ch == '\0') {
774        mr_msg(6, "%s already in tree", original_string);
775        return (1);
776    }
777
778    // add here
779    newnode = (struct s_node *) mr_malloc(sizeof(struct s_node));
780    if (char_to_add < node->ch) // add to the left of node
781    {
782        mr_msg(7, "depth=%d char=%c --- adding (left)", depth,
783                char_to_add);
784        memcpy((void *) newnode, (void *) node, sizeof(struct s_node));
785        node->right = newnode;
786    } else if (char_to_add > node->ch)  // add to the right of node
787    {
788        mr_msg(7, "depth=%d char=%c --- adding (right)", depth,
789                char_to_add);
790        newnode->right = node->right;   // newnode is to the RIGHT of node
791        node->right = newnode;
792        node = newnode;
793    }
794    // from now on, we're working on 'node'
795    node->down = NULL;
796    node->ch = char_to_add;
797    node->expanded = node->selected = FALSE;
798    if (char_to_add == '\0') {
799        mr_msg(6, "Added %s OK", original_string);
800        return (0);
801    }
802    // add the rest
803    mr_msg(6, "Adding remaining chars ('%s')", string_to_add + 1);
804    for (i = 1; i < noof_chars; i++) {
805        node->down = (struct s_node *) mr_malloc(sizeof(struct s_node));
806        node = node->down;
807        char_to_add = string_to_add[i];
808        mr_msg(6, "Adding '%c'", char_to_add);
809        node->ch = char_to_add;
810        node->right = node->down = NULL;
811        node->expanded = node->selected = FALSE;
812        if (!node->ch) {
813            node->selected = sosodef;
814        }
815    }
816    mr_msg(6, "Finally - added %s OK", original_string);
817    return (0);
818}
819
820
821/**
822 * Load a filelist into a <tt>struct s_node</tt>.
823 * When you are done with the filelist, call free_filelist().
824 * @param filelist_fname The file to load the filelist from.
825 * @return A filelist tree structure.
826 */
827struct s_node *load_filelist(char *filelist_fname)
828{
829
830    struct s_node *filelist = NULL;
831    FILE *pin = NULL;
832    char *command_to_open_fname = NULL;
833    char *fname = NULL;
834    char *tmp = NULL;
835    int pos_in_fname = 0;
836    size_t n = 0;
837    int percentage = 0;
838
839    long lines_in_filelist = 0L;
840    long lino = 0L;
841
842    assert_string_is_neither_NULL_nor_zerolength(filelist_fname);
843
844    if (!does_file_exist(filelist_fname)) {
845        fatal_error("filelist does not exist -- cannot load it");
846    }
847    log_to_screen(_("Loading filelist"));
848    mr_asprintf(&command_to_open_fname, "gzip -dc %s", filelist_fname);
849    mr_asprintf(&tmp, "zcat %s | wc -l", filelist_fname);
850    mr_msg(6, "tmp = %s", tmp);
851    lines_in_filelist =
852        atol(call_program_and_get_last_line_of_output(tmp));
853    if (lines_in_filelist < 3) {
854        log_to_screen(_("Warning - surprisingly short filelist."));
855    }
856    g_original_noof_lines_in_filelist = lines_in_filelist;
857    filelist = (struct s_node *) mr_malloc(sizeof(struct s_node));
858    filelist->ch = '/';
859    filelist->right = NULL;
860    filelist->down = mr_malloc(sizeof(struct s_node));
861    filelist->expanded = filelist->selected = FALSE;
862    (filelist->down)->ch = '\0';
863    (filelist->down)->right = (filelist->down)->down = FALSE;
864    (filelist->down)->expanded = (filelist->down)->selected = FALSE;
865    if (!(pin = popen(command_to_open_fname, "r"))) {
866        log_OS_error("Unable to openin filelist_fname");
867        return (NULL);
868    }
869    mr_free(command_to_open_fname);
870
871    open_evalcall_form(_("Loading filelist from disk"));
872    for (mr_getline(&fname, &n, pin); !feof(pin); mr_getline(&fname, &n, pin)) {
873        if ((fname[strlen(fname) - 1] == 13
874             || fname[strlen(fname) - 1] == 10) && strlen(fname) > 0) {
875            fname[strlen(fname) - 1] = '\0';
876        }
877        if (!strlen(fname)) {
878            continue;
879        }
880        for (pos_in_fname = 0; fname[pos_in_fname] != '\0'; pos_in_fname++) {
881            if (fname[pos_in_fname] != '/') {
882                continue;
883            }
884            mr_asprintf(&tmp, fname);
885            tmp[pos_in_fname] = '\0';
886            if (strlen(tmp)) {
887                add_string_at_node(filelist, tmp);
888            }
889            mr_free(tmp);
890        }
891        add_string_at_node(filelist, fname);
892        if (!(++lino % 1111) && (lines_in_filelist != 0)) {
893            percentage = (int) (lino * 100 / lines_in_filelist);
894            update_evalcall_form(percentage);
895        }
896    }
897    mr_free(fname);
898    paranoid_pclose(pin);
899    close_evalcall_form();
900    log_it("Finished loading filelist");
901    return (filelist);
902}
903
904
905/**
906 * Log a list of files in @p node.
907 * @param node The toplevel node to use.
908 */
909void show_filelist(struct s_node *node)
910{
911    static int depth = 0;
912    static char current_string[200];
913
914    if (depth == 0) {
915        mr_msg(0, "----------------show filelist--------------");
916    }
917    current_string[depth] = node->ch;
918
919    mr_msg(3, "depth=%d", depth);
920    if (node->down) {
921        mr_msg(3, "moving down");
922        depth++;
923        show_filelist(node->down);
924        depth--;
925    }
926
927    if (!node->ch) {
928        mr_msg(0, "%s\n", current_string);
929    }
930
931    if (node->right) {
932        mr_msg(3, "moving right");
933        show_filelist(node->right);
934    }
935    if (depth == 0) {
936        mr_msg(0, "----------------show filelist--------------");
937    }
938    return;
939}
940
941
942/**
943 * Reset the filelist to the state it was when it was loaded. This does not
944 * touch the file on disk.
945 * @param filelist The filelist tree structure.
946 */
947void reload_filelist(struct s_node *filelist)
948{
949    assert(filelist != NULL);
950    toggle_node_selection(filelist, FALSE);
951    toggle_path_expandability(filelist, "/", FALSE);
952    toggle_all_root_dirs_on(filelist);
953}
954
955
956/**
957 * Save a filelist tree structure to disk.
958 * @param filelist The filelist tree structure to save.
959 * @param outfname Where to save it.
960 */
961void save_filelist(struct s_node *filelist, char *outfname)
962{
963    /*@ int ********************************************************* */
964    static int percentage;
965    static int depth = 0;
966
967    /*@ buffers ***************************************************** */
968    static char str[MAX_STR_LEN];
969
970    /*@ structures ************************************************** */
971    struct s_node *node;
972
973    /*@ pointers **************************************************** */
974    static FILE *fout = NULL;
975
976    /*@ long ******************************************************** */
977    static long lines_in_filelist = 0;
978    static long lino = 0;
979
980    /*@ end vars *************************************************** */
981
982    assert(filelist != NULL);
983    assert(outfname != NULL);   // will be zerolength if save_filelist() is called by itself
984    if (depth == 0) {
985        log_to_screen(_("Saving filelist"));
986        if (!(fout = fopen(outfname, "w"))) {
987            fatal_error("Cannot openout/save filelist");
988        }
989        lines_in_filelist = g_original_noof_lines_in_filelist;  /* set by load_filelist() */
990        open_evalcall_form(_("Saving selection to disk"));
991    }
992    for (node = filelist; node != NULL; node = node->right) {
993        str[depth] = node->ch;
994        mr_msg(5, "depth=%d ch=%c", depth, node->ch);
995        if (!node->ch) {
996//    if (node->selected)
997//      {
998            fprintf(fout, "%s\n", str);
999//      }
1000            if (!(++lino % 1111) && (lines_in_filelist != 0)) {
1001                percentage = (int) (lino * 100 / lines_in_filelist);
1002                update_evalcall_form(percentage);
1003            }
1004        }
1005        if (node->down) {
1006            depth++;
1007            save_filelist(node->down, "");
1008            depth--;
1009        }
1010    }
1011    if (depth == 0) {
1012        paranoid_fclose(fout);
1013        close_evalcall_form();
1014        log_it("Finished saving filelist");
1015    }
1016}
1017
1018
1019/**
1020 * Toggle all root dirs on.
1021 * @param filelist The filelist tree structure to operate on.
1022 * @bug I don't understand this function. Would someone care to explain it?
1023 */
1024void toggle_all_root_dirs_on(struct s_node *filelist)
1025{
1026    /*@ structures ************************************************** */
1027    struct s_node *node;
1028
1029    /*@ int ********************************************************* */
1030    static int depth = 0;
1031    static int root_dirs_expanded;
1032
1033    /*@ buffers ***************************************************** */
1034    static char filename[MAX_STR_LEN];
1035
1036    /*@ end vars *************************************************** */
1037
1038    assert(filelist != NULL);
1039    if (depth == 0) {
1040        log_it("Toggling all root dirs ON");
1041        root_dirs_expanded = 0;
1042    }
1043    for (node = filelist; node != NULL; node = node->right) {
1044        filename[depth] = node->ch;
1045        if (node->ch == '\0' && strlen(filename) > 1
1046            && (!strchr(filename + 1, '/'))) {
1047            node->selected = FALSE;
1048            node->expanded = TRUE;
1049//    log_it (filename);
1050            root_dirs_expanded++;
1051        }
1052        if (node->down) {
1053            depth++;
1054            toggle_all_root_dirs_on(node->down);
1055            depth--;
1056        }
1057    }
1058    if (depth == 0) {
1059        log_it("Finished toggling all root dirs ON");
1060    }
1061}
1062
1063
1064/**
1065 * Toggle the expandability of a path.
1066 * @param filelist The filelist tree to operate on.
1067 * @param pathname The path to toggle expandability of.
1068 * @param on_or_off Whether to toggle it on or off.
1069 * @bug I don't understand this function. Would someone care to explain it?
1070 */
1071void
1072toggle_path_expandability(struct s_node *filelist, char *pathname,
1073                          bool on_or_off)
1074{
1075
1076    /*@ int ******************************************************** */
1077    static int depth = 0;
1078    static int total_expanded;
1079    static int root_depth;
1080    int j;
1081    /*@ structures ************************************************* */
1082    struct s_node *node;
1083
1084    /*@ buffers **************************************************** */
1085    static char current_filename[MAX_STR_LEN];
1086
1087    /*@ end vars *************************************************** */
1088
1089    assert(filelist != NULL);
1090    assert_string_is_neither_NULL_nor_zerolength(pathname);
1091    if (depth == 0) {
1092        total_expanded = 0;
1093//      log_it ("Toggling path's expandability");
1094        for (root_depth = (int) strlen(pathname);
1095             root_depth > 0 && pathname[root_depth - 1] != '/';
1096             root_depth--);
1097        if (root_depth < 2) {
1098            root_depth = (int) strlen(pathname);
1099        }
1100    }
1101    for (node = filelist; node != NULL; node = node->right) {
1102        current_filename[depth] = node->ch;
1103        if (node->down) {
1104            depth++;
1105            toggle_path_expandability(node->down, pathname, on_or_off);
1106            depth--;
1107        }
1108        if (node->ch == '\0') {
1109            if (!strncmp(pathname, current_filename, strlen(pathname))) {
1110                for (j = root_depth;
1111                     current_filename[j] != '/'
1112                     && current_filename[j] != '\0'; j++);
1113                if (current_filename[j] != '\0') {
1114                    for (j++;
1115                         current_filename[j] != '/'
1116                         && current_filename[j] != '\0'; j++);
1117                }
1118                if (current_filename[j] == '\0') {
1119                    node->expanded =
1120                        (!strcmp(pathname, current_filename) ? TRUE :
1121                         on_or_off);
1122                }
1123            }
1124        }
1125        if (node->expanded) {
1126            if (total_expanded < ARBITRARY_MAXIMUM - 32
1127                || !strrchr(current_filename + strlen(pathname), '/')) {
1128                total_expanded++;
1129            } else {
1130                node->expanded = FALSE;
1131            }
1132        }
1133    }
1134    if (depth == 0) {
1135//      log_it ("Finished toggling expandability");
1136    }
1137}
1138
1139
1140/**
1141 * Toggle whether a path is selected.
1142 * @param filelist The filelist tree to operate on.
1143 * @param pathname The path to toggle selection of.
1144 * @param on_or_off Whether to toggle it on or off.
1145 * @bug I don't understand this function. Would someone care to explain it?
1146 */
1147void
1148toggle_path_selection(struct s_node *filelist, char *pathname,
1149                      bool on_or_off)
1150{
1151    /*@ int ********************************************************* */
1152    static int depth = 0;
1153    int j;
1154
1155    /*@ structures ************************************************** */
1156    struct s_node *node;
1157
1158    /*@ buffers ***************************************************** */
1159    static char current_filename[MAX_STR_LEN];
1160
1161    /*@ end vars *************************************************** */
1162    assert(filelist != NULL);
1163    assert_string_is_neither_NULL_nor_zerolength(pathname);
1164    if (depth == 0) {
1165        log_it("Toggling path's selection");
1166    }
1167    for (node = filelist; node != NULL; node = node->right) {
1168        current_filename[depth] = node->ch;
1169        if (node->down) {
1170            depth++;
1171            toggle_path_selection(node->down, pathname, on_or_off);
1172            depth--;
1173        }
1174        if (node->ch == '\0') {
1175            if (!strncmp(pathname, current_filename, strlen(pathname))) {
1176                for (j = 0;
1177                     pathname[j] != '\0'
1178                     && pathname[j] == current_filename[j]; j++);
1179                if (current_filename[j] == '/'
1180                    || current_filename[j] == '\0') {
1181                    node->selected = on_or_off;
1182                }
1183            }
1184        }
1185    }
1186    if (depth == 0) {
1187        log_it("Finished toggling selection");
1188    }
1189}
1190
1191
1192/**
1193 * Toggle node selection of a filelist tree.
1194 * @param filelist The filelist tree to operate on.
1195 * @param on_or_off Whether to toggle selection on or off.
1196 * @bug I don't understand this function. Would someone care to explain it?
1197 */
1198void toggle_node_selection(struct s_node *filelist, bool on_or_off)
1199{
1200    /*@ structure ************************************************** */
1201    struct s_node *node;
1202
1203    /*@ end vars *************************************************** */
1204    assert(filelist != NULL);
1205    for (node = filelist; node != NULL; node = node->right) {
1206        if (node->ch == '/') {
1207            continue;
1208        }                       /* don't go deep */
1209        if (node->ch == '\0') {
1210            node->selected = on_or_off;
1211        }
1212        if (node->down) {
1213            toggle_node_selection(node->down, on_or_off);
1214        }
1215    }
1216}
1217
1218
1219/**
1220 * The pathname to the skeleton filelist, used to give better progress reporting for mondo_makefilelist().
1221 */
1222char *g_skeleton_filelist = NULL;
1223
1224/**
1225 * Number of entries in the skeleton filelist.
1226 */
1227long g_skeleton_entries = 0;
1228
1229/**
1230 * Wrapper around mondo_makefilelist().
1231 * @param bkpinfo The backup information structure. Fields used:
1232 * - @c bkpinfo->differential
1233 * - @c bkpinfo->exclude_paths
1234 * - @c bkpinfo->include_paths
1235 * - @c bkpinfo->make_filelist
1236 * - @c bkpinfo->scratchdir
1237 * - @c bkpinfo->tmpdir
1238 * @return 0 for success, nonzero for failure.
1239 * @see mondo_makefilelist
1240 */
1241int prepare_filelist()
1242{
1243
1244    /*@ int **************************************************** */
1245    int res = 0;
1246
1247    assert(bkpinfo != NULL);
1248    log_it("tmpdir=%s; scratchdir=%s", bkpinfo->tmpdir,
1249           bkpinfo->scratchdir);
1250    if (bkpinfo->make_filelist) {
1251        mvaddstr_and_log_it(g_currentY, 0,
1252                            _("Making catalog of files to be backed up"));
1253    } else {
1254        mvaddstr_and_log_it(g_currentY, 0,
1255                            _("Using supplied catalog of files to be backed up"));
1256    }
1257
1258    if (bkpinfo->make_filelist) {
1259        res =
1260            mondo_makefilelist(MONDO_LOGFILE, bkpinfo->tmpdir,
1261                               bkpinfo->scratchdir, bkpinfo->include_paths,
1262                               bkpinfo->exclude_paths,
1263                               bkpinfo->differential, NULL);
1264    } else {
1265        res =
1266            mondo_makefilelist(MONDO_LOGFILE, bkpinfo->tmpdir,
1267                               bkpinfo->scratchdir, NULL,
1268                               bkpinfo->exclude_paths,
1269                               bkpinfo->differential,
1270                               bkpinfo->include_paths);
1271    }
1272
1273    if (res) {
1274        log_OS_error("Call to mondo_makefilelist failed");
1275        mvaddstr_and_log_it(g_currentY++, 74, _("Failed."));
1276    } else {
1277        mvaddstr_and_log_it(g_currentY++, 74, _("Done."));
1278    }
1279    return (res);
1280}
1281
1282
1283/**
1284 * Recursively list all files in @p dir newer than @p time_of_last_full_backup to @p fout.
1285 * @param dir The directory to list to @p fout.
1286 * @param sth The directories to skip (exclude).
1287 * @param fout The file to write everything to.
1288 * @param time_of_last_full_backup Only backup files newer than this (0 to disable).
1289 * @return 0, always.
1290 * @bug Return value should be @c void.
1291 */
1292int open_and_list_dir(char *dir, char *sth, FILE * fout,
1293                      time_t time_of_last_full_backup)
1294{
1295    const char delims[] = " ";
1296    DIR *dip = NULL;
1297    struct dirent *dit = NULL;
1298
1299    struct stat statbuf;
1300    char new[MAX_STR_LEN];
1301    char *sth_B = NULL;
1302    static int percentage = 0;
1303    char *ith_B = NULL;
1304    char *skip_these = NULL;
1305    char *new_with_spaces = NULL;
1306    static char *name_of_evalcall_form = NULL;
1307    int i = 0;
1308    char *strtmp = NULL;
1309    char *token = NULL;
1310    char *find_excludes = NULL;
1311    int lastpos = 0;
1312    static int depth = 0;
1313    char *p = NULL;
1314    static int counter = 0;
1315    static int uberctr = 0;
1316    static char *find_skeleton_marker = NULL;
1317    static long skeleton_lino = 0;
1318    static time_t last_time = 0;
1319    time_t this_time;
1320
1321    sth_B = malloc(8*MAX_STR_LEN);
1322    malloc_string(ith_B);
1323    malloc_string(new_with_spaces);
1324    p = strrchr(dir, '/');
1325    if (p) {
1326        if (!strcmp(p, "/.") || !strcmp(p, "/..")) {
1327            return (0);
1328        }
1329    }
1330
1331    if (!depth) {
1332        malloc_string(name_of_evalcall_form);
1333        malloc_string(find_skeleton_marker);
1334        mr_asprintf(&find_excludes, " ");
1335        while((token = mr_strtok (sth, delims, &lastpos))) {
1336          mr_asprintf(&strtmp,"%s", find_excludes);
1337          mr_free(find_excludes);
1338          mr_asprintf(&find_excludes,"%s -path %s -prune -o", strtmp, token);
1339          mr_free(strtmp);
1340          mr_free(token);
1341        }
1342#if linux
1343        // 2.6 has /sys as a proc-type thing -- must be excluded
1344        mr_asprintf(&strtmp,
1345             "find %s -maxdepth %d -fstype mvfs -prune -o -path /dev/shm -prune -o %s -type d -print > %s 2> /dev/null",
1346             dir, MAX_SKEL_DEPTH, find_excludes, g_skeleton_filelist);
1347#else
1348        // On BSD, for example, /sys is the kernel sources -- don't exclude
1349        mr_asprintf(&strtmp,
1350             "find %s -maxdepth %d -fstype mvfs -prune -o -path /proc -prune -o %s -type d -print > %s 2> /dev/null",
1351                dir, MAX_SKEL_DEPTH, find_excludes, g_skeleton_filelist);
1352#endif
1353        mr_free(find_excludes);
1354        mr_msg(5, "find command = %s", strtmp);
1355        system(strtmp);
1356        mr_free(strtmp);
1357        mr_asprintf(&strtmp, "wc -l %s | awk '{print $1;}'", g_skeleton_filelist);
1358        g_skeleton_entries =
1359            1 + atol(call_program_and_get_last_line_of_output(strtmp));
1360        mr_free(strtmp);
1361        sprintf(name_of_evalcall_form, "Making catalog of %s", dir);
1362        open_evalcall_form(name_of_evalcall_form);
1363        find_skeleton_marker[0] = '\0';
1364        skeleton_lino = 1;
1365        mr_msg(5, "entries = %ld", g_skeleton_entries);
1366        percentage = 0;
1367    } else if (depth <= MAX_SKEL_DEPTH) // update evalcall form if appropriate
1368    {
1369        sprintf(find_skeleton_marker,
1370                "grep -Fv \"%s\" %s > %s.new 2> /dev/null", dir,
1371                g_skeleton_filelist, g_skeleton_filelist);
1372        if (!system(find_skeleton_marker) && (g_skeleton_entries != 0)) {
1373            percentage = (int) (skeleton_lino * 100 / g_skeleton_entries);
1374            skeleton_lino++;
1375            sprintf(find_skeleton_marker, "mv -f %s.new %s",
1376                    g_skeleton_filelist, g_skeleton_filelist);
1377            run_program_and_log_output(find_skeleton_marker, 8);
1378            time(&this_time);
1379            if (this_time != last_time) {
1380                last_time = this_time;
1381#ifndef _XWIN
1382                if (!g_text_mode) {
1383                    mr_asprintf(&strtmp, "Reading %-68s", dir);
1384                    newtDrawRootText(0, g_noof_rows - 3, strtmp);
1385                    mr_free(strtmp);
1386                }
1387#endif
1388                update_evalcall_form(percentage);
1389            }
1390        }
1391    }
1392
1393    depth++;
1394
1395    if (sth[0] == ' ') {
1396        skip_these = sth;
1397    } else {
1398        skip_these = sth_B;
1399        sprintf(skip_these, " %s ", sth);
1400    }
1401    sprintf(new_with_spaces, " %s ", dir);
1402    if ((dip = opendir(dir)) == NULL) {
1403        mr_asprintf(&strtmp, "opendir %s", dir);
1404        log_OS_error(strtmp);
1405        mr_free(strtmp);
1406    } else if (strstr(skip_these, new_with_spaces)) {
1407        fprintf(fout, "%s\n", dir); // if excluded dir then print dir ONLY
1408    } else {
1409        fprintf(fout, "%s\n", dir);
1410        while ((dit = readdir(dip)) != NULL) {
1411            i++;
1412            strcpy(new, dir);
1413            if (strcmp(dir, "/")) {
1414                strcat(new, "/");
1415            }
1416            strcat(new, dit->d_name);
1417            new_with_spaces[0] = ' ';
1418            strcpy(new_with_spaces + 1, new);
1419            strcat(new_with_spaces, " ");
1420            if (strstr(skip_these, new_with_spaces)) {
1421                fprintf(fout, "%s\n", new);
1422            } else {
1423                if (!lstat(new, &statbuf)) {
1424                    if (!S_ISLNK(statbuf.st_mode)
1425                        && S_ISDIR(statbuf.st_mode)) {
1426                        open_and_list_dir(new, skip_these, fout,
1427                                          time_of_last_full_backup);
1428                    } else {
1429                        if (time_of_last_full_backup == 0
1430                            || time_of_last_full_backup <
1431                            statbuf.st_ctime) {
1432                            fprintf(fout, "%s\n", new);
1433                            if ((counter++) > 128) {
1434                                counter = 0;
1435                                uberctr++;
1436#ifndef _XWIN
1437                                mr_asprintf(&strtmp, " %c ",
1438                                        special_dot_char(uberctr));
1439                                if (!g_text_mode) {
1440                                    newtDrawRootText(77, g_noof_rows - 3,
1441                                                     strtmp);
1442                                    newtRefresh();
1443                                }
1444                                mr_free(strtmp);
1445#endif
1446                            }
1447                        }
1448                    }
1449                }
1450            }
1451        }
1452    }
1453    if (dip) {
1454        if (closedir(dip) == -1) {
1455            log_OS_error("closedir");
1456        }
1457    }
1458    depth--;
1459    if (!depth) {
1460        close_evalcall_form();
1461        mr_free(name_of_evalcall_form);
1462        mr_free(find_skeleton_marker);
1463        unlink(g_skeleton_filelist);
1464        mr_msg(5, "g_skeleton_entries = %ld", g_skeleton_entries);
1465    }
1466    mr_free(sth_B);
1467    mr_free(ith_B);
1468    mr_free(new_with_spaces);
1469    return (0);
1470}
1471
1472
1473/**
1474 * Get the next entry in the space-separated list in @p incoming.
1475 * So if @p incoming was '"one and two" three four', we would
1476 * return "one and two".
1477 * @param incoming The list to get the next entry from.
1478 * @return The first item in the list (respecting double quotes).
1479 * @note The returned string points to static data that will be overwritten with each call.
1480 */
1481char *next_entry(char *incoming)
1482{
1483    static char sz_res[MAX_STR_LEN];
1484    char *p = NULL;
1485    bool in_quotes = FALSE;
1486
1487    strcpy(sz_res, incoming);
1488    p = sz_res;
1489    while ((*p != ' ' || in_quotes) && *p != '\0') {
1490        if (*p == '\"') {
1491            in_quotes = !in_quotes;
1492        }
1493        p++;
1494    }
1495    *p = '\0';
1496    return (sz_res);
1497}
1498
1499
1500/**
1501 * Create the filelist for the backup. It will be stored in [scratchdir]/archives/filelist.full.
1502 * @param logfile Unused.
1503 * @param tmpdir The tmpdir of the backup.
1504 * @param scratchdir The scratchdir of the backup.
1505 * @param include_paths The paths to back up, or NULL if you're using a user-defined filelist.
1506 * @param excp The paths to NOT back up.
1507 * @param differential The differential level (currently only 0 and 1 are supported).
1508 * @param userdef_filelist The user-defined filelist, or NULL if you're using @p include_paths.
1509 * @return 0, always.
1510 * @bug @p logfile is unused.
1511 * @bug Return value is meaningless.
1512 */
1513int mondo_makefilelist(char *logfile, char *tmpdir, char *scratchdir,
1514                       char *include_paths, char *excp, bool differential,
1515                       char *userdef_filelist)
1516{
1517    char sz_datefile_wildcard[] = MONDO_CACHE"/difflevel.%d";
1518    char *p = NULL;
1519    char *q = NULL;
1520
1521    char *sz_datefile = NULL;
1522    char *sz_filelist = NULL;
1523    char *exclude_paths = NULL;
1524    int i = 0;
1525    FILE *fout = NULL;
1526    char *command = NULL;
1527    time_t time_of_last_full_backup = 0;
1528    struct stat statbuf;
1529
1530    malloc_string(sz_datefile);
1531    if (!(exclude_paths = malloc(8*MAX_STR_LEN))) {
1532        fatal_error("Cannot malloc exclude_paths");
1533    }
1534    malloc_string(g_skeleton_filelist);
1535    // The pathname to the skeleton filelist, used to give better progress reporting for mondo_makefilelist().
1536    mr_asprintf(&sz_datefile, sz_datefile_wildcard, 0);
1537    if (!include_paths && !userdef_filelist) {
1538        fatal_error
1539            ("Please supply either include_paths or userdef_filelist");
1540    }
1541// make hole for filelist
1542    mr_asprintf(&command, "mkdir -p %s/archives", scratchdir);
1543    paranoid_system(command);
1544    mr_free(command);
1545
1546    mr_asprintf(&sz_filelist, "%s/tmpfs/filelist.full", tmpdir);
1547    make_hole_for_file(sz_filelist);
1548
1549    if (! differential) {
1550        // restore last good datefile if it exists
1551        mr_asprintf(&command, "cp -f %s.aborted %s", sz_datefile,
1552                 sz_datefile);
1553        run_program_and_log_output(command, 3);
1554        mr_free(command);
1555
1556        // backup last known good datefile just in case :)
1557        if (does_file_exist(sz_datefile)) {
1558            mr_asprintf(&command, "mv -f %s %s.aborted", sz_datefile,
1559                     sz_datefile);
1560            paranoid_system(command);
1561            mr_free(command);
1562        }
1563        make_hole_for_file(sz_datefile);
1564        write_one_liner_data_file(sz_datefile,
1565                                  call_program_and_get_last_line_of_output
1566                                  ("date +%s"));
1567    } else if (lstat(sz_datefile, &statbuf)) {
1568        mr_msg(2,
1569                "Warning - unable to find date of previous backup. Full backup instead.");
1570        differential = FALSE;
1571        time_of_last_full_backup = 0;
1572    } else {
1573        time_of_last_full_backup = statbuf.st_mtime;
1574        mr_msg(2, "Differential backup. Yay.");
1575    }
1576    mr_free(sz_datefile);
1577
1578// use user-specified filelist (if specified)
1579    if (userdef_filelist) {
1580        mr_msg(1,
1581                "Using the user-specified filelist - %s - instead of calculating one",
1582                userdef_filelist);
1583        mr_asprintf(&command, "cp -f %s %s", userdef_filelist, sz_filelist);
1584        if (run_program_and_log_output(command, 3)) {
1585            fatal_error("Failed to copy user-specified filelist");
1586        }
1587        mr_free(command);
1588    } else {
1589        mr_msg(2, "include_paths = '%s'", include_paths);
1590        mr_msg(1, "Calculating filelist");
1591        snprintf(exclude_paths, (size_t)8*MAX_STR_LEN," %s %s %s %s %s %s %s . .. \
1592" MNT_CDROM " /mnt/floppy /media \
1593/proc /sys /tmp /var/log/lastlog /root/images/mondo " MINDI_CACHE " " MONDO_CACHE, excp, call_program_and_get_last_line_of_output("locate /win386.swp 2> /dev/null"), call_program_and_get_last_line_of_output("locate /hiberfil.sys 2> /dev/null"), call_program_and_get_last_line_of_output("locate /pagefile.sys 2> /dev/null"), (tmpdir[0] == '/' && tmpdir[1] == '/') ? (tmpdir + 1) : tmpdir, (scratchdir[0] == '/' && scratchdir[1] == '/') ? (scratchdir + 1) : scratchdir);
1594
1595        mr_msg(2, "Excluding paths = '%s'", exclude_paths);
1596        mr_msg(2,
1597                "Generating skeleton filelist so that we can track our progress");
1598        sprintf(g_skeleton_filelist, "%s/tmpfs/skeleton.txt", tmpdir);
1599        make_hole_for_file(g_skeleton_filelist);
1600        mr_msg(4, "g_skeleton_entries = %ld", g_skeleton_entries);
1601        mr_msg(2, "Opening out filelist to %s", sz_filelist);
1602        if (!(fout = fopen(sz_filelist, "w"))) {
1603            fatal_error("Cannot openout to sz_filelist");
1604        }
1605        i = 0;
1606        if (strlen(include_paths) == 0) {
1607            mr_msg(1, "Including only '/' in %s", sz_filelist);
1608            open_and_list_dir("/", exclude_paths, fout,
1609                              time_of_last_full_backup);
1610        } else {
1611            p = include_paths;
1612            while (*p) {
1613                q = next_entry(p);
1614                mr_msg(1, "Including %s in filelist %s", q, sz_filelist);
1615                open_and_list_dir(q, exclude_paths, fout,
1616                                  time_of_last_full_backup);
1617                p += strlen(q);
1618                while (*p == ' ') {
1619                    p++;
1620                }
1621            }
1622        }
1623        paranoid_fclose(fout);
1624        mr_free(exclude_paths);
1625    }
1626    mr_msg(2, "Copying new filelist to scratchdir");
1627    mr_asprintf(&command, "mkdir -p %s/archives", scratchdir);
1628    paranoid_system(command);
1629    mr_free(command);
1630
1631    mr_asprintf(&command, "cp -f %s %s/archives/", sz_filelist, scratchdir);
1632    paranoid_system(command);
1633    mr_free(command);
1634
1635    mr_asprintf(&command, "mv -f %s %s", sz_filelist, tmpdir);
1636    paranoid_system(command);
1637    mr_free(command);
1638    mr_free(sz_filelist);
1639    mr_free(g_skeleton_filelist);
1640    mr_msg(2, "Exiting");
1641    return (0);
1642}
1643
1644
1645/**
1646 * Locate the string @p string_to_find in the tree rooted at @p startnode.
1647 * @param startnode The node containing the root of the directory tree.
1648 * @param string_to_find The string to look for at @p startnode.
1649 * @return The node containing the last element of @p string_to_find, or NULL if
1650 * it was not found.
1651 */
1652struct s_node *find_string_at_node(struct s_node *startnode,
1653                                   char *string_to_find)
1654{
1655    /*@ int ******************************************************** */
1656    int noof_chars;
1657    static int depth = 0;
1658    static char original_string[MAX_STR_LEN];
1659
1660    /*@ sturctures ************************************************* */
1661    struct s_node *node;
1662
1663    /*@ char  ****************************************************** */
1664    char char_to_find;
1665
1666    /*@ bools ****************************************************** */
1667
1668    if (!depth) {
1669        strcpy(original_string, string_to_find);
1670    }
1671
1672    assert(startnode != NULL);
1673    assert(string_to_find != NULL);
1674
1675    noof_chars = strlen(string_to_find) + 1;    /* we include the '\0' */
1676
1677    mr_msg(7, "starting --- str=%s", string_to_find);
1678
1679/* walk across tree if necessary */
1680    node = startnode;
1681    char_to_find = string_to_find[0];
1682    if (node->right != NULL && node->ch < char_to_find) {
1683        mr_msg(7, "depth=%d --- going RIGHT ... %c-->%c", depth,
1684                char_to_find, node->ch, (node->right)->ch);
1685        return (find_string_at_node(node->right, string_to_find));
1686    }
1687
1688/* walk down tree if appropriate */
1689    if (node->down != NULL && node->ch == char_to_find) {
1690        mr_msg(7, "depth=%d char=%c --- going DOWN", depth, char_to_find);
1691        depth++;
1692        node = find_string_at_node(node->down, string_to_find + 1);
1693        depth--;
1694        return (node);
1695    }
1696
1697    if (char_to_find == '\0' && node->ch == '\0') {
1698        mr_msg(7, "%s is in tree", original_string);
1699        return (node);
1700    } else {
1701        mr_msg(7, "%s is NOT in tree", original_string);
1702        return (NULL);
1703    }
1704}
1705
1706
1707/**
1708 * Write all entries in @p needles_list_fname which are also in
1709 * @p filelist to @p matches_list_fname.
1710 * @param needles_list_fname A file containing strings to look for, 1 per line.
1711 * @param filelist The node for the root of the directory structure to search in.
1712 * @param matches_list_fname The filename where we should put the matches.
1713 * @return The number of matches found.
1714 */
1715long save_filelist_entries_in_common(char *needles_list_fname,
1716                                     struct s_node *filelist,
1717                                     char *matches_list_fname,
1718                                     bool use_star)
1719{
1720    int retval = 0;
1721    struct s_node *found_node = NULL;
1722    FILE *fin = NULL;
1723    FILE *fout = NULL;
1724    char *fname = NULL;
1725    char *tmp = NULL;
1726    size_t len = 0;             // Scrub's patch doesn't work without that
1727
1728    mr_msg(5, "starting");
1729    mr_msg(5, "needles_list_fname = %s", needles_list_fname);
1730    mr_msg(5, "matches_list_fname = %s", matches_list_fname);
1731    if (!(fin = fopen(needles_list_fname, "r"))) {
1732        fatal_error("Cannot openin needles_list_fname");
1733    }
1734    if (!(fout = fopen(matches_list_fname, "w"))) {
1735        fatal_error("Cannot openout matches_list_fname");
1736    }
1737    while (!feof(fin)) {
1738        mr_getline(&fname, &len, fin);
1739        if (!use_star) {
1740            if (fname[0] == '/') {
1741                mr_asprintf(&tmp, fname);
1742            } else {
1743                mr_asprintf(&tmp, "/%s", fname);
1744            }
1745            mr_free(fname);
1746            fname = tmp;
1747        }
1748        while (strlen(fname) > 0 && fname[strlen(fname) - 1] < 32) {
1749            fname[strlen(fname) - 1] = '\0';
1750        }
1751
1752        mr_msg(5, "Looking for '%s'", fname);
1753        found_node = find_string_at_node(filelist, fname);
1754        if (found_node) {
1755            if (found_node->selected) {
1756                if (fname[0] == '/') {
1757                    mr_asprintf(&tmp, fname + 1);
1758                    mr_free(fname);
1759                    fname = tmp;
1760                }
1761                mr_msg(5, "Found '%s'", fname);
1762                tmp = mr_stresc(fname, WILDCHARS, BACKSLASH);
1763                fprintf(fout, "%s\n", tmp);
1764                mr_free(tmp);
1765                retval++;
1766            }
1767        }
1768        mr_free(fname);
1769    }
1770    paranoid_fclose(fout);
1771    paranoid_fclose(fin);
1772    return (retval);
1773}
1774
1775
1776/**
1777 * Add all files listed in @p list_of_files_fname to the directory structure rooted at
1778 * @p filelist.
1779 * @param filelist The top node of the directory structure to add the files to.
1780 * @param list_of_files_fname The file containing the files to add, 1 per line.
1781 * @param flag_em If TRUE, then flag the added files for restoration.
1782 * @return 0 for success, nonzero for failure.
1783 */
1784int add_list_of_files_to_filelist(struct s_node *filelist,
1785                                  char *list_of_files_fname, bool flag_em)
1786{
1787    FILE *fin;
1788    char *tmp = NULL;
1789    size_t n = 0;
1790    struct s_node *nod;
1791
1792    mr_msg(3, "Adding %s to filelist", list_of_files_fname);
1793    if (!(fin = fopen(list_of_files_fname, "r"))) {
1794        iamhere(list_of_files_fname);
1795        return (1);
1796    }
1797    for (mr_getline(&tmp, &n, fin); !feof(fin); mr_getline(&tmp, &n, fin)) {
1798        if (!tmp[0]) {
1799            continue;
1800        }
1801        if ((tmp[strlen(tmp) - 1] == 13 || tmp[strlen(tmp) - 1] == 10)
1802            && strlen(tmp) > 0) {
1803            tmp[strlen(tmp) - 1] = '\0';
1804        }
1805        mr_msg(2, "tmp = '%s'", tmp);
1806        if (!tmp[0]) {
1807            continue;
1808        }
1809        if ((nod = find_string_at_node(filelist, tmp))) {
1810            mr_msg(5, "Found '%s' in filelist already. Cool.", tmp);
1811        } else {
1812            add_string_at_node(filelist, tmp);
1813            nod = find_string_at_node(filelist, tmp);
1814        }
1815
1816        if (nod && flag_em) {
1817            toggle_path_selection(filelist, tmp, TRUE);
1818            mr_msg(5, "Flagged '%s'", tmp);
1819        }
1820    }
1821    paranoid_fclose(fin);
1822    mr_free(tmp);
1823    return (0);
1824}
1825
1826/* @} - end of filelistGroup */
Note: See TracBrowser for help on using the repository browser.