source: branches/stable/mindi-busybox/networking/wget.c @ 1770

Last change on this file since 1770 was 1770, checked in by Bruno Cornec, 13 years ago
  • Better output for mindi-busybox revision
  • Remove dummy file created on NFS - report from Arnaud Tiger <arnaud.tiger_at_hp.com>
  • strace useful for debug
  • fix new versions for pb (2.0.0 for mindi and 1.7.2 for mindi-busybox)
  • fix build process for mindi-busybox + options used in that version (dd for label-partitions-as-necessary)
  • fix typo in label-partitions-as-necessary which doesn't seem to work
  • Update to busybox 1.7.2
  • perl is now required at restore time to support uuid swap partitions (and will be used for many other thigs

in the future for sure)

  • next mindi version will be 2.0.0 due to all the changes made in it (udev may break working distros)
  • small optimization in mindi on keyboard handling (one single find instead of multiple)
  • better interaction for USB device when launching mindi manually
  • attempt to automatically guess block disk size for ramdisk
  • fix typos in bkphw
  • Fix the remaining problem with UUID support for swap partitions
  • Updates mondoarchive man page for USB support
  • Adds preliminary Hardware support to mindi (Proliant SSSTK)
  • Tries to add udev support also for rhel4
  • Fix UUID support which was still broken.
  • Be conservative in test for the start-nfs script
  • Update config file for mindi-busybox for 1.7.2 migration
  • Try to run around a busybox bug (1.2.2 pb on inexistant links)
  • Add build content for mindi-busybox in pb
  • Remove distributions content for mindi-busybox
  • Fix a warning on inexistant raidtab
  • Solve problem on tmpfs in restore init (Problem of inexistant symlink and busybox)
  • Create MONDO_CACHE and use it everywhere + creation at start
  • Really never try to eject a USB device
  • Fix a issue with &> usage (replaced with 1> and 2>)
  • Adds magic file to depllist in order to have file working + ldd which helps for debugging issues
  • tty modes correct to avoid sh error messages
  • Use ext3 normally and not ext2 instead
  • USB device should be corrected after reading (take 1st part)
  • Adds a mount_USB_here function derived from mount_CDROM_here
  • usb detection place before /dev detection in device name at restore time
  • Fix when restoring from USB: media is asked in interactive mode
  • Adds USB support for mondorestore
  • mount_cdrom => mount_media
  • elilo.efi is now searched throughout /boot/efi and not in a fixed place as there is no standard
  • untar-and-softlink => untar (+ interface change)
  • suppress useless softlinks creation/removal in boot process
  • avoids udevd messages on groups
  • Increase # of disks to 99 as in mindi at restore time (should be a conf file parameter)
  • skip existing big file creation
  • seems to work correctly for USB mindi boot
  • Adds group and tty link to udev conf
  • Always load usb-torage (even 2.6) to initiate USB bus discovery
  • Better printing of messages
  • Attempt to fix a bug in supporting OpenSusE 10.3 kernel for initramfs (mindi may now use multiple regex for kernel initrd detection)
  • Links were not correctly done as non relative for modules in mindi
  • exclusion of modules denied now works
  • Also create modules in their ordinary place, so that classical modprobe works + copy modules.dep
  • Fix bugs for DENY_MODS handling
  • Add device /dev/console for udev
  • ide-generic should now really be excluded
  • Fix a bug in major number for tty
  • If udev then adds modprobe/insmod to rootfs
  • tty0 is also cretaed with udev
  • ide-generic put rather in DENY_MODS
  • udevd remove from deplist s handled in mindi directly
  • better default for mindi when using --usb
  • Handles dynamically linked busybox (in case we want to use it soon ;-)
  • Adds fixed devices to create for udev
  • ide-generic should not be part of the initrd when using libata v2
  • support a dynamically linked udev (case on Ubuntu 7.10 and Mandriva 2008.0 so should be quite generic) This will give incitation to move to dyn. linked binaries in the initrd which will help for other tasks (ia6 4)
  • Improvement in udev support (do not use cl options not available in busybox)
  • Udev in mindi
    • auto creation of the right links at boot time with udev-links.conf(from Mandriva 2008.0)
    • rework startup of udev as current makes kernel crash (from Mandriva 2008.0)
    • add support for 64 bits udev
  • Try to render MyInsmod? silent at boot time
  • Adds udev support (mandatory for newest distributions to avoid remapping of devices in a different way as on the original system)
  • We also need vaft format support for USB boot
  • Adds libusual support (Ubuntu 7.10 needs it for USB)
  • Improve Ubuntu/Debian? keyboard detection and support
  • pbinit adapted to new pb (0.8.10). Filtering of docs done in it
  • Suppress some mondo warnings and errors on USB again
  • Tries to fix lack of files in deb mindi package
  • Verify should now work for USB devices
  • More log/mesages improvement for USB support
  • - Supress g_erase_tmpdir_and_scratchdir
  • Improve some log messages for USB support
  • Try to improve install in mindi to avoid issues with isolinux.cfg not installed vene if in the pkg :-(
  • Improve mindi-busybox build
  • In conformity with pb 0.8.9
  • Add support for Ubuntu 7.10 in build process
  • Add USB Key button to Menu UI (CD streamer removed)
  • Attempt to fix error messages on tmp/scratch files at the end by removing those dir at the latest possible.
  • Fix a bug linked to the size of the -E param which could be used (Arnaud Tiger/René? Ribaud).
  • Integrate ~/.pbrc content into mondorescue.pb (required project-builder >= 0.8.7)
  • Put mondorescue in conformity with new pb filtering rules
  • Add USB support at restore time (no test done yet). New start-usb script PB varibale added where useful
  • Unmounting USB device before removal of temporary scratchdir
  • Stil refining USB copy back to mondo (one command was not executed)
  • No need to have the image subdor in the csratchdir when USB.
  • umount the USB partition before attempting to use it
  • Remove useless copy from mindi to mondo at end of USB handling

(risky merge, we are raising the limits of 2 diverging branches. The status of stable is not completely sure as such. Will need lots of tests, but it's not yet done :-()
(merge -r1692:1769 $SVN_M/branches/2.2.5)

File size: 21.7 KB
Line 
1/* vi: set sw=4 ts=4: */
2/*
3 * wget - retrieve a file using HTTP or FTP
4 *
5 * Chip Rosenthal Covad Communications <chip@laserlink.net>
6 *
7 */
8
9/* We want libc to give us xxx64 functions also */
10/* http://www.unix.org/version2/whatsnew/lfs20mar.html */
11//#define _LARGEFILE64_SOURCE 1
12
13#include <getopt.h> /* for struct option */
14#include "libbb.h"
15
16struct host_info {
17    // May be used if we ever will want to free() all xstrdup()s...
18    /* char *allocated; */
19    char *host;
20    int port;
21    char *path;
22    int is_ftp;
23    char *user;
24};
25
26static void parse_url(char *url, struct host_info *h);
27static FILE *open_socket(len_and_sockaddr *lsa);
28static char *gethdr(char *buf, size_t bufsiz, FILE *fp, int *istrunc);
29static int ftpcmd(const char *s1, const char *s2, FILE *fp, char *buf);
30
31/* Globals (can be accessed from signal handlers */
32static off_t content_len;        /* Content-length of the file */
33static off_t beg_range;          /* Range at which continue begins */
34#if ENABLE_FEATURE_WGET_STATUSBAR
35static off_t transferred;        /* Number of bytes transferred so far */
36#endif
37static bool chunked;                     /* chunked transfer encoding */
38#if ENABLE_FEATURE_WGET_STATUSBAR
39static void progressmeter(int flag);
40static const char *curfile;             /* Name of current file being transferred */
41enum {
42    STALLTIME = 5                   /* Seconds when xfer considered "stalled" */
43};
44#else
45static ALWAYS_INLINE void progressmeter(int flag) {}
46#endif
47
48/* Read NMEMB bytes into PTR from STREAM.  Returns the number of bytes read,
49 * and a short count if an eof or non-interrupt error is encountered.  */
50static size_t safe_fread(void *ptr, size_t nmemb, FILE *stream)
51{
52    size_t ret;
53    char *p = (char*)ptr;
54
55    do {
56        clearerr(stream);
57        ret = fread(p, 1, nmemb, stream);
58        p += ret;
59        nmemb -= ret;
60    } while (nmemb && ferror(stream) && errno == EINTR);
61
62    return p - (char*)ptr;
63}
64
65/* Read a line or SIZE-1 bytes into S, whichever is less, from STREAM.
66 * Returns S, or NULL if an eof or non-interrupt error is encountered.  */
67static char *safe_fgets(char *s, int size, FILE *stream)
68{
69    char *ret;
70
71    do {
72        clearerr(stream);
73        ret = fgets(s, size, stream);
74    } while (ret == NULL && ferror(stream) && errno == EINTR);
75
76    return ret;
77}
78
79#if ENABLE_FEATURE_WGET_AUTHENTICATION
80/* Base64-encode character string. buf is assumed to be char buf[512]. */
81static char *base64enc_512(char buf[512], const char *str)
82{
83    unsigned len = strlen(str);
84    if (len > 512/4*3 - 10) /* paranoia */
85        len = 512/4*3 - 10;
86    bb_uuencode(buf, str, len, bb_uuenc_tbl_base64);
87    return buf;
88}
89#endif
90
91int wget_main(int argc, char **argv);
92int wget_main(int argc, char **argv)
93{
94    char buf[512];
95    struct host_info server, target;
96    len_and_sockaddr *lsa;
97    int n, status;
98    int port;
99    int try = 5;
100    unsigned opt;
101    char *str;
102    char *proxy = 0;
103    char *dir_prefix = NULL;
104#if ENABLE_FEATURE_WGET_LONG_OPTIONS
105    char *extra_headers = NULL;
106    llist_t *headers_llist = NULL;
107#endif
108
109    FILE *sfp = NULL;               /* socket to web/ftp server         */
110    FILE *dfp = NULL;               /* socket to ftp server (data)      */
111    char *fname_out = NULL;         /* where to direct output (-O)      */
112    bool got_clen = 0;              /* got content-length: from server  */
113    int output_fd = -1;
114    bool use_proxy = 1;             /* Use proxies if env vars are set  */
115    const char *proxy_flag = "on";  /* Use proxies if env vars are set  */
116    const char *user_agent = "Wget";/* "User-Agent" header field        */
117    static const char keywords[] ALIGN1 =
118        "content-length\0""transfer-encoding\0""chunked\0""location\0";
119    enum {
120        KEY_content_length = 1, KEY_transfer_encoding, KEY_chunked, KEY_location
121    };
122    enum {
123        WGET_OPT_CONTINUE   = 0x1,
124        WGET_OPT_SPIDER     = 0x2,
125        WGET_OPT_QUIET      = 0x4,
126        WGET_OPT_OUTNAME    = 0x8,
127        WGET_OPT_PREFIX     = 0x10,
128        WGET_OPT_PROXY      = 0x20,
129        WGET_OPT_USER_AGENT = 0x40,
130        WGET_OPT_PASSIVE    = 0x80,
131        WGET_OPT_HEADER     = 0x100,
132    };
133#if ENABLE_FEATURE_WGET_LONG_OPTIONS
134    static const char wget_longopts[] ALIGN1 =
135        /* name, has_arg, val */
136        "continue\0"         No_argument       "c"
137        "spider\0"           No_argument       "s"
138        "quiet\0"            No_argument       "q"
139        "output-document\0"  Required_argument "O"
140        "directory-prefix\0" Required_argument "P"
141        "proxy\0"            Required_argument "Y"
142        "user-agent\0"       Required_argument "U"
143        "passive-ftp\0"      No_argument       "\xff"
144        "header\0"           Required_argument "\xfe"
145        ;
146    applet_long_options = wget_longopts;
147#endif
148    /* server.allocated = target.allocated = NULL; */
149    opt_complementary = "-1" USE_FEATURE_WGET_LONG_OPTIONS(":\xfe::");
150    opt = getopt32(argv, "csqO:P:Y:U:",
151                &fname_out, &dir_prefix,
152                &proxy_flag, &user_agent
153                USE_FEATURE_WGET_LONG_OPTIONS(, &headers_llist)
154                );
155    if (strcmp(proxy_flag, "off") == 0) {
156        /* Use the proxy if necessary */
157        use_proxy = 0;
158    }
159#if ENABLE_FEATURE_WGET_LONG_OPTIONS
160    if (headers_llist) {
161        int size = 1;
162        char *cp;
163        llist_t *ll = headers_llist;
164        while (ll) {
165            size += strlen(ll->data) + 2;
166            ll = ll->link;
167        }
168        extra_headers = cp = xmalloc(size);
169        while (headers_llist) {
170            cp += sprintf(cp, "%s\r\n", headers_llist->data);
171            headers_llist = headers_llist->link;
172        }
173    }
174#endif
175
176    parse_url(argv[optind], &target);
177    server.host = target.host;
178    server.port = target.port;
179
180    /* Use the proxy if necessary */
181    if (use_proxy) {
182        proxy = getenv(target.is_ftp ? "ftp_proxy" : "http_proxy");
183        if (proxy && *proxy) {
184            parse_url(proxy, &server);
185        } else {
186            use_proxy = 0;
187        }
188    }
189
190    /* Guess an output filename */
191    if (!fname_out) {
192        // Dirty hack. Needed because bb_get_last_path_component
193        // will destroy trailing / by storing '\0' in last byte!
194        if (!last_char_is(target.path, '/')) {
195            fname_out = bb_get_last_path_component(target.path);
196#if ENABLE_FEATURE_WGET_STATUSBAR
197            curfile = fname_out;
198#endif
199        }
200        if (!fname_out || !fname_out[0]) {
201            /* bb_get_last_path_component writes
202             * to last '/' only. We don't have one here... */
203            fname_out = (char*)"index.html";
204#if ENABLE_FEATURE_WGET_STATUSBAR
205            curfile = fname_out;
206#endif
207        }
208        if (dir_prefix != NULL)
209            fname_out = concat_path_file(dir_prefix, fname_out);
210#if ENABLE_FEATURE_WGET_STATUSBAR
211    } else {
212        curfile = bb_get_last_path_component(fname_out);
213#endif
214    }
215    /* Impossible?
216    if ((opt & WGET_OPT_CONTINUE) && !fname_out)
217        bb_error_msg_and_die("cannot specify continue (-c) without a filename (-O)"); */
218
219    /* Determine where to start transfer */
220    if (LONE_DASH(fname_out)) {
221        output_fd = 1;
222        opt &= ~WGET_OPT_CONTINUE;
223    }
224    if (opt & WGET_OPT_CONTINUE) {
225        output_fd = open(fname_out, O_WRONLY);
226        if (output_fd >= 0) {
227            beg_range = xlseek(output_fd, 0, SEEK_END);
228        }
229        /* File doesn't exist. We do not create file here yet.
230           We are not sure it exists on remove side */
231    }
232
233    /* We want to do exactly _one_ DNS lookup, since some
234     * sites (i.e. ftp.us.debian.org) use round-robin DNS
235     * and we want to connect to only one IP... */
236    lsa = xhost2sockaddr(server.host, server.port);
237    if (!(opt & WGET_OPT_QUIET)) {
238        fprintf(stderr, "Connecting to %s (%s)\n", server.host,
239                xmalloc_sockaddr2dotted(&lsa->sa));
240        /* We leak result of xmalloc_sockaddr2dotted */
241    }
242
243    if (use_proxy || !target.is_ftp) {
244        /*
245         *  HTTP session
246         */
247        do {
248            got_clen = chunked = 0;
249
250            if (!--try)
251                bb_error_msg_and_die("too many redirections");
252
253            /* Open socket to http server */
254            if (sfp) fclose(sfp);
255            sfp = open_socket(lsa);
256
257            /* Send HTTP request.  */
258            if (use_proxy) {
259                fprintf(sfp, "GET %stp://%s/%s HTTP/1.1\r\n",
260                    target.is_ftp ? "f" : "ht", target.host,
261                    target.path);
262            } else {
263                fprintf(sfp, "GET /%s HTTP/1.1\r\n", target.path);
264            }
265
266            fprintf(sfp, "Host: %s\r\nUser-Agent: %s\r\n",
267                target.host, user_agent);
268
269#if ENABLE_FEATURE_WGET_AUTHENTICATION
270            if (target.user) {
271                fprintf(sfp, "Proxy-Authorization: Basic %s\r\n"+6,
272                    base64enc_512(buf, target.user));
273            }
274            if (use_proxy && server.user) {
275                fprintf(sfp, "Proxy-Authorization: Basic %s\r\n",
276                    base64enc_512(buf, server.user));
277            }
278#endif
279
280            if (beg_range)
281                fprintf(sfp, "Range: bytes=%"OFF_FMT"d-\r\n", beg_range);
282#if ENABLE_FEATURE_WGET_LONG_OPTIONS
283            if (extra_headers)
284                fputs(extra_headers, sfp);
285#endif
286            fprintf(sfp, "Connection: close\r\n\r\n");
287
288            /*
289            * Retrieve HTTP response line and check for "200" status code.
290            */
291 read_response:
292            if (fgets(buf, sizeof(buf), sfp) == NULL)
293                bb_error_msg_and_die("no response from server");
294
295            str = buf;
296            str = skip_non_whitespace(str);
297            str = skip_whitespace(str);
298            // FIXME: no error check
299            // xatou wouldn't work: "200 OK"
300            status = atoi(str);
301            switch (status) {
302            case 0:
303            case 100:
304                while (gethdr(buf, sizeof(buf), sfp, &n) != NULL)
305                    /* eat all remaining headers */;
306                goto read_response;
307            case 200:
308                break;
309            case 300:   /* redirection */
310            case 301:
311            case 302:
312            case 303:
313                break;
314            case 206:
315                if (beg_range)
316                    break;
317                /*FALLTHRU*/
318            default:
319                /* Show first line only and kill any ESC tricks */
320                buf[strcspn(buf, "\n\r\x1b")] = '\0';
321                bb_error_msg_and_die("server returned error: %s", buf);
322            }
323
324            /*
325             * Retrieve HTTP headers.
326             */
327            while ((str = gethdr(buf, sizeof(buf), sfp, &n)) != NULL) {
328                /* gethdr did already convert the "FOO:" string to lowercase */
329                smalluint key = index_in_strings(keywords, *&buf) + 1;
330                if (key == KEY_content_length) {
331                    content_len = BB_STRTOOFF(str, NULL, 10);
332                    if (errno || content_len < 0) {
333                        bb_error_msg_and_die("content-length %s is garbage", str);
334                    }
335                    got_clen = 1;
336                    continue;
337                }
338                if (key == KEY_transfer_encoding) {
339                    if (index_in_strings(keywords, str_tolower(str)) + 1 != KEY_chunked)
340                        bb_error_msg_and_die("server wants to do %s transfer encoding", str);
341                    chunked = got_clen = 1;
342                }
343                if (key == KEY_location) {
344                    if (str[0] == '/')
345                        /* free(target.allocated); */
346                        target.path = /* target.allocated = */ xstrdup(str+1);
347                    else {
348                        parse_url(str, &target);
349                        if (use_proxy == 0) {
350                            server.host = target.host;
351                            server.port = target.port;
352                        }
353                        free(lsa);
354                        lsa = xhost2sockaddr(server.host, server.port);
355                        break;
356                    }
357                }
358            }
359        } while (status >= 300);
360
361        dfp = sfp;
362
363    } else {
364
365        /*
366         *  FTP session
367         */
368        if (!target.user)
369            target.user = xstrdup("anonymous:busybox@");
370
371        sfp = open_socket(lsa);
372        if (ftpcmd(NULL, NULL, sfp, buf) != 220)
373            bb_error_msg_and_die("%s", buf+4);
374
375        /*
376         * Splitting username:password pair,
377         * trying to log in
378         */
379        str = strchr(target.user, ':');
380        if (str)
381            *(str++) = '\0';
382        switch (ftpcmd("USER ", target.user, sfp, buf)) {
383        case 230:
384            break;
385        case 331:
386            if (ftpcmd("PASS ", str, sfp, buf) == 230)
387                break;
388            /* FALLTHRU (failed login) */
389        default:
390            bb_error_msg_and_die("ftp login: %s", buf+4);
391        }
392
393        ftpcmd("TYPE I", NULL, sfp, buf);
394
395        /*
396         * Querying file size
397         */
398        if (ftpcmd("SIZE ", target.path, sfp, buf) == 213) {
399            content_len = BB_STRTOOFF(buf+4, NULL, 10);
400            if (errno || content_len < 0) {
401                bb_error_msg_and_die("SIZE value is garbage");
402            }
403            got_clen = 1;
404        }
405
406        /*
407         * Entering passive mode
408         */
409        if (ftpcmd("PASV", NULL, sfp, buf) != 227) {
410 pasv_error:
411            bb_error_msg_and_die("bad response to %s: %s", "PASV", buf);
412        }
413        // Response is "227 garbageN1,N2,N3,N4,P1,P2[)garbage]
414        // Server's IP is N1.N2.N3.N4 (we ignore it)
415        // Server's port for data connection is P1*256+P2
416        str = strrchr(buf, ')');
417        if (str) str[0] = '\0';
418        str = strrchr(buf, ',');
419        if (!str) goto pasv_error;
420        port = xatou_range(str+1, 0, 255);
421        *str = '\0';
422        str = strrchr(buf, ',');
423        if (!str) goto pasv_error;
424        port += xatou_range(str+1, 0, 255) * 256;
425        set_nport(lsa, htons(port));
426        dfp = open_socket(lsa);
427
428        if (beg_range) {
429            sprintf(buf, "REST %"OFF_FMT"d", beg_range);
430            if (ftpcmd(buf, NULL, sfp, buf) == 350)
431                content_len -= beg_range;
432        }
433
434        if (ftpcmd("RETR ", target.path, sfp, buf) > 150)
435            bb_error_msg_and_die("bad response to RETR: %s", buf);
436    }
437    if (opt & WGET_OPT_SPIDER) {
438        if (ENABLE_FEATURE_CLEAN_UP)
439            fclose(sfp);
440        goto done;
441    }
442
443    /*
444     * Retrieve file
445     */
446    if (chunked) {
447        fgets(buf, sizeof(buf), dfp);
448        content_len = STRTOOFF(buf, NULL, 16);
449        /* FIXME: error check?? */
450    }
451
452    /* Do it before progressmeter (want to have nice error message) */
453    if (output_fd < 0)
454        output_fd = xopen(fname_out,
455            O_WRONLY|O_CREAT|O_EXCL|O_TRUNC);
456
457    if (!(opt & WGET_OPT_QUIET))
458        progressmeter(-1);
459
460    do {
461        while (content_len > 0 || !got_clen) {
462            unsigned rdsz = sizeof(buf);
463            if (content_len < sizeof(buf) && (chunked || got_clen))
464                rdsz = (unsigned)content_len;
465            n = safe_fread(buf, rdsz, dfp);
466            if (n <= 0)
467                break;
468            if (full_write(output_fd, buf, n) != n) {
469                bb_perror_msg_and_die(bb_msg_write_error);
470            }
471#if ENABLE_FEATURE_WGET_STATUSBAR
472            transferred += n;
473#endif
474            if (got_clen) {
475                content_len -= n;
476            }
477        }
478
479        if (chunked) {
480            safe_fgets(buf, sizeof(buf), dfp); /* This is a newline */
481            safe_fgets(buf, sizeof(buf), dfp);
482            content_len = STRTOOFF(buf, NULL, 16);
483            /* FIXME: error check? */
484            if (content_len == 0) {
485                chunked = 0; /* all done! */
486            }
487        }
488
489        if (n == 0 && ferror(dfp)) {
490            bb_perror_msg_and_die(bb_msg_read_error);
491        }
492    } while (chunked);
493
494    if (!(opt & WGET_OPT_QUIET))
495        progressmeter(1);
496
497    if ((use_proxy == 0) && target.is_ftp) {
498        fclose(dfp);
499        if (ftpcmd(NULL, NULL, sfp, buf) != 226)
500            bb_error_msg_and_die("ftp error: %s", buf+4);
501        ftpcmd("QUIT", NULL, sfp, buf);
502    }
503done:
504    exit(EXIT_SUCCESS);
505}
506
507
508static void parse_url(char *src_url, struct host_info *h)
509{
510    char *url, *p, *sp;
511
512    /* h->allocated = */ url = xstrdup(src_url);
513
514    if (strncmp(url, "http://", 7) == 0) {
515        h->port = bb_lookup_port("http", "tcp", 80);
516        h->host = url + 7;
517        h->is_ftp = 0;
518    } else if (strncmp(url, "ftp://", 6) == 0) {
519        h->port = bb_lookup_port("ftp", "tcp", 21);
520        h->host = url + 6;
521        h->is_ftp = 1;
522    } else
523        bb_error_msg_and_die("not an http or ftp url: %s", url);
524
525    // FYI:
526    // "Real" wget 'http://busybox.net?var=a/b' sends this request:
527    //   'GET /?var=a/b HTTP 1.0'
528    //   and saves 'index.html?var=a%2Fb' (we save 'b')
529    // wget 'http://busybox.net?login=john@doe':
530    //   request: 'GET /?login=john@doe HTTP/1.0'
531    //   saves: 'index.html?login=john@doe' (we save '?login=john@doe')
532    // wget 'http://busybox.net#test/test':
533    //   request: 'GET / HTTP/1.0'
534    //   saves: 'index.html' (we save 'test')
535    //
536    // We also don't add unique .N suffix if file exists...
537    sp = strchr(h->host, '/');
538    p = strchr(h->host, '?'); if (!sp || (p && sp > p)) sp = p;
539    p = strchr(h->host, '#'); if (!sp || (p && sp > p)) sp = p;
540    if (!sp) {
541        /* must be writable because of bb_get_last_path_component() */
542        static char nullstr[] ALIGN1 = "";
543        h->path = nullstr;
544    } else if (*sp == '/') {
545        *sp = '\0';
546        h->path = sp + 1;
547    } else { // '#' or '?'
548        // http://busybox.net?login=john@doe is a valid URL
549        // memmove converts to:
550        // http:/busybox.nett?login=john@doe...
551        memmove(h->host-1, h->host, sp - h->host);
552        h->host--;
553        sp[-1] = '\0';
554        h->path = sp;
555    }
556
557    sp = strrchr(h->host, '@');
558    h->user = NULL;
559    if (sp != NULL) {
560        h->user = h->host;
561        *sp = '\0';
562        h->host = sp + 1;
563    }
564
565    sp = h->host;
566}
567
568
569static FILE *open_socket(len_and_sockaddr *lsa)
570{
571    FILE *fp;
572
573    /* glibc 2.4 seems to try seeking on it - ??! */
574    /* hopefully it understands what ESPIPE means... */
575    fp = fdopen(xconnect_stream(lsa), "r+");
576    if (fp == NULL)
577        bb_perror_msg_and_die("fdopen");
578
579    return fp;
580}
581
582
583static char *gethdr(char *buf, size_t bufsiz, FILE *fp, int *istrunc)
584{
585    char *s, *hdrval;
586    int c;
587
588    *istrunc = 0;
589
590    /* retrieve header line */
591    if (fgets(buf, bufsiz, fp) == NULL)
592        return NULL;
593
594    /* see if we are at the end of the headers */
595    for (s = buf; *s == '\r'; ++s)
596        ;
597    if (s[0] == '\n')
598        return NULL;
599
600    /* convert the header name to lower case */
601    for (s = buf; isalnum(*s) || *s == '-'; ++s)
602        *s = tolower(*s);
603
604    /* verify we are at the end of the header name */
605    if (*s != ':')
606        bb_error_msg_and_die("bad header line: %s", buf);
607
608    /* locate the start of the header value */
609    for (*s++ = '\0'; *s == ' ' || *s == '\t'; ++s)
610        ;
611    hdrval = s;
612
613    /* locate the end of header */
614    while (*s != '\0' && *s != '\r' && *s != '\n')
615        ++s;
616
617    /* end of header found */
618    if (*s != '\0') {
619        *s = '\0';
620        return hdrval;
621    }
622
623    /* Rats!  The buffer isn't big enough to hold the entire header value. */
624    while (c = getc(fp), c != EOF && c != '\n')
625        ;
626    *istrunc = 1;
627    return hdrval;
628}
629
630static int ftpcmd(const char *s1, const char *s2, FILE *fp, char *buf)
631{
632    int result;
633    if (s1) {
634        if (!s2) s2 = "";
635        fprintf(fp, "%s%s\r\n", s1, s2);
636        fflush(fp);
637    }
638
639    do {
640        char *buf_ptr;
641
642        if (fgets(buf, 510, fp) == NULL) {
643            bb_perror_msg_and_die("error getting response");
644        }
645        buf_ptr = strstr(buf, "\r\n");
646        if (buf_ptr) {
647            *buf_ptr = '\0';
648        }
649    } while (!isdigit(buf[0]) || buf[3] != ' ');
650
651    buf[3] = '\0';
652    result = xatoi_u(buf);
653    buf[3] = ' ';
654    return result;
655}
656
657#if ENABLE_FEATURE_WGET_STATUSBAR
658/* Stuff below is from BSD rcp util.c, as added to openshh.
659 * Original copyright notice is retained at the end of this file.
660 */
661static int
662getttywidth(void)
663{
664    int width;
665    get_terminal_width_height(0, &width, NULL);
666    return width;
667}
668
669static void
670updateprogressmeter(int ignore)
671{
672    int save_errno = errno;
673
674    progressmeter(0);
675    errno = save_errno;
676}
677
678static void alarmtimer(int iwait)
679{
680    struct itimerval itv;
681
682    itv.it_value.tv_sec = iwait;
683    itv.it_value.tv_usec = 0;
684    itv.it_interval = itv.it_value;
685    setitimer(ITIMER_REAL, &itv, NULL);
686}
687
688static void
689progressmeter(int flag)
690{
691    static unsigned lastupdate_sec;
692    static unsigned start_sec;
693    static off_t lastsize, totalsize;
694
695    off_t abbrevsize;
696    unsigned since_last_update, elapsed;
697    unsigned ratio;
698    int barlength, i;
699
700    if (flag == -1) { /* first call to progressmeter */
701        start_sec = monotonic_sec();
702        lastupdate_sec = start_sec;
703        lastsize = 0;
704        totalsize = content_len + beg_range; /* as content_len changes.. */
705    }
706
707    ratio = 100;
708    if (totalsize != 0 && !chunked) {
709        /* long long helps to have it working even if !LFS */
710        ratio = (unsigned) (100ULL * (transferred+beg_range) / totalsize);
711        if (ratio > 100) ratio = 100;
712    }
713
714    fprintf(stderr, "\r%-20.20s%4d%% ", curfile, ratio);
715
716    barlength = getttywidth() - 49;
717    if (barlength > 0) {
718        /* god bless gcc for variable arrays :) */
719        i = barlength * ratio / 100;
720        {
721            char buf[i+1];
722            memset(buf, '*', i);
723            buf[i] = '\0';
724            fprintf(stderr, "|%s%*s|", buf, barlength - i, "");
725        }
726    }
727    i = 0;
728    abbrevsize = transferred + beg_range;
729    while (abbrevsize >= 100000) {
730        i++;
731        abbrevsize >>= 10;
732    }
733    /* see http://en.wikipedia.org/wiki/Tera */
734    fprintf(stderr, "%6d%c ", (int)abbrevsize, " kMGTPEZY"[i]);
735
736// Nuts! Ain't it easier to update progress meter ONLY when we transferred++?
737// FIXME: get rid of alarmtimer + updateprogressmeter mess
738
739    elapsed = monotonic_sec();
740    since_last_update = elapsed - lastupdate_sec;
741    if (transferred > lastsize) {
742        lastupdate_sec = elapsed;
743        lastsize = transferred;
744        if (since_last_update >= STALLTIME) {
745            /* We "cut off" these seconds from elapsed time
746             * by adjusting start time */
747            start_sec += since_last_update;
748        }
749        since_last_update = 0; /* we are un-stalled now */
750    }
751    elapsed -= start_sec; /* now it's "elapsed since start" */
752
753    if (since_last_update >= STALLTIME) {
754        fprintf(stderr, " - stalled -");
755    } else {
756        off_t to_download = totalsize - beg_range;
757        if (transferred <= 0 || (int)elapsed <= 0 || transferred > to_download || chunked) {
758            fprintf(stderr, "--:--:-- ETA");
759        } else {
760            /* to_download / (transferred/elapsed) - elapsed: */
761            int eta = (int) ((unsigned long long)to_download*elapsed/transferred - elapsed);
762            /* (long long helps to have working ETA even if !LFS) */
763            i = eta % 3600;
764            fprintf(stderr, "%02d:%02d:%02d ETA", eta / 3600, i / 60, i % 60);
765        }
766    }
767
768    if (flag == -1) { /* first call to progressmeter */
769        struct sigaction sa;
770        sa.sa_handler = updateprogressmeter;
771        sigemptyset(&sa.sa_mask);
772        sa.sa_flags = SA_RESTART;
773        sigaction(SIGALRM, &sa, NULL);
774        alarmtimer(1);
775    } else if (flag == 1) { /* last call to progressmeter */
776        alarmtimer(0);
777        transferred = 0;
778        putc('\n', stderr);
779    }
780}
781#endif /* FEATURE_WGET_STATUSBAR */
782
783/* Original copyright notice which applies to the CONFIG_FEATURE_WGET_STATUSBAR stuff,
784 * much of which was blatantly stolen from openssh.  */
785
786/*-
787 * Copyright (c) 1992, 1993
788 *  The Regents of the University of California.  All rights reserved.
789 *
790 * Redistribution and use in source and binary forms, with or without
791 * modification, are permitted provided that the following conditions
792 * are met:
793 * 1. Redistributions of source code must retain the above copyright
794 *    notice, this list of conditions and the following disclaimer.
795 * 2. Redistributions in binary form must reproduce the above copyright
796 *    notice, this list of conditions and the following disclaimer in the
797 *    documentation and/or other materials provided with the distribution.
798 *
799 * 3. <BSD Advertising Clause omitted per the July 22, 1999 licensing change
800 *      ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change>
801 *
802 * 4. Neither the name of the University nor the names of its contributors
803 *    may be used to endorse or promote products derived from this software
804 *    without specific prior written permission.
805 *
806 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
807 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
808 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
809 * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
810 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
811 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
812 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
813 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
814 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
815 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
816 * SUCH DAMAGE.
817 *
818 */
Note: See TracBrowser for help on using the repository browser.