source: MondoRescue/branches/stable/mindi-busybox/networking/wget.c@ 1842

Last change on this file since 1842 was 1770, checked in by Bruno Cornec, 16 years ago
  • Better output for mindi-busybox revision
  • Remove dummy file created on NFS - report from Arnaud Tiger <arnaud.tiger_at_hp.com>
  • strace useful for debug
  • fix new versions for pb (2.0.0 for mindi and 1.7.2 for mindi-busybox)
  • fix build process for mindi-busybox + options used in that version (dd for label-partitions-as-necessary)
  • fix typo in label-partitions-as-necessary which doesn't seem to work
  • Update to busybox 1.7.2
  • perl is now required at restore time to support uuid swap partitions (and will be used for many other thigs

in the future for sure)

  • next mindi version will be 2.0.0 due to all the changes made in it (udev may break working distros)
  • small optimization in mindi on keyboard handling (one single find instead of multiple)
  • better interaction for USB device when launching mindi manually
  • attempt to automatically guess block disk size for ramdisk
  • fix typos in bkphw
  • Fix the remaining problem with UUID support for swap partitions
  • Updates mondoarchive man page for USB support
  • Adds preliminary Hardware support to mindi (Proliant SSSTK)
  • Tries to add udev support also for rhel4
  • Fix UUID support which was still broken.
  • Be conservative in test for the start-nfs script
  • Update config file for mindi-busybox for 1.7.2 migration
  • Try to run around a busybox bug (1.2.2 pb on inexistant links)
  • Add build content for mindi-busybox in pb
  • Remove distributions content for mindi-busybox
  • Fix a warning on inexistant raidtab
  • Solve problem on tmpfs in restore init (Problem of inexistant symlink and busybox)
  • Create MONDO_CACHE and use it everywhere + creation at start
  • Really never try to eject a USB device
  • Fix a issue with &> usage (replaced with 1> and 2>)
  • Adds magic file to depllist in order to have file working + ldd which helps for debugging issues
  • tty modes correct to avoid sh error messages
  • Use ext3 normally and not ext2 instead
  • USB device should be corrected after reading (take 1st part)
  • Adds a mount_USB_here function derived from mount_CDROM_here
  • usb detection place before /dev detection in device name at restore time
  • Fix when restoring from USB: media is asked in interactive mode
  • Adds USB support for mondorestore
  • mount_cdrom => mount_media
  • elilo.efi is now searched throughout /boot/efi and not in a fixed place as there is no standard
  • untar-and-softlink => untar (+ interface change)
  • suppress useless softlinks creation/removal in boot process
  • avoids udevd messages on groups
  • Increase # of disks to 99 as in mindi at restore time (should be a conf file parameter)
  • skip existing big file creation
  • seems to work correctly for USB mindi boot
  • Adds group and tty link to udev conf
  • Always load usb-torage (even 2.6) to initiate USB bus discovery
  • Better printing of messages
  • Attempt to fix a bug in supporting OpenSusE 10.3 kernel for initramfs (mindi may now use multiple regex for kernel initrd detection)
  • Links were not correctly done as non relative for modules in mindi
  • exclusion of modules denied now works
  • Also create modules in their ordinary place, so that classical modprobe works + copy modules.dep
  • Fix bugs for DENY_MODS handling
  • Add device /dev/console for udev
  • ide-generic should now really be excluded
  • Fix a bug in major number for tty
  • If udev then adds modprobe/insmod to rootfs
  • tty0 is also cretaed with udev
  • ide-generic put rather in DENY_MODS
  • udevd remove from deplist s handled in mindi directly
  • better default for mindi when using --usb
  • Handles dynamically linked busybox (in case we want to use it soon ;-)
  • Adds fixed devices to create for udev
  • ide-generic should not be part of the initrd when using libata v2
  • support a dynamically linked udev (case on Ubuntu 7.10 and Mandriva 2008.0 so should be quite generic) This will give incitation to move to dyn. linked binaries in the initrd which will help for other tasks (ia6 4)
  • Improvement in udev support (do not use cl options not available in busybox)
  • Udev in mindi
    • auto creation of the right links at boot time with udev-links.conf(from Mandriva 2008.0)
    • rework startup of udev as current makes kernel crash (from Mandriva 2008.0)
    • add support for 64 bits udev
  • Try to render MyInsmod silent at boot time
  • Adds udev support (mandatory for newest distributions to avoid remapping of devices in a different way as on the original system)
  • We also need vaft format support for USB boot
  • Adds libusual support (Ubuntu 7.10 needs it for USB)
  • Improve Ubuntu/Debian keyboard detection and support
  • pbinit adapted to new pb (0.8.10). Filtering of docs done in it
  • Suppress some mondo warnings and errors on USB again
  • Tries to fix lack of files in deb mindi package
  • Verify should now work for USB devices
  • More log/mesages improvement for USB support
  • - Supress g_erase_tmpdir_and_scratchdir
  • Improve some log messages for USB support
  • Try to improve install in mindi to avoid issues with isolinux.cfg not installed vene if in the pkg :-(
  • Improve mindi-busybox build
  • In conformity with pb 0.8.9
  • Add support for Ubuntu 7.10 in build process
  • Add USB Key button to Menu UI (CD streamer removed)
  • Attempt to fix error messages on tmp/scratch files at the end by removing those dir at the latest possible.
  • Fix a bug linked to the size of the -E param which could be used (Arnaud Tiger/René Ribaud).
  • Integrate ~/.pbrc content into mondorescue.pb (required project-builder >= 0.8.7)
  • Put mondorescue in conformity with new pb filtering rules
  • Add USB support at restore time (no test done yet). New start-usb script PB varibale added where useful
  • Unmounting USB device before removal of temporary scratchdir
  • Stil refining USB copy back to mondo (one command was not executed)
  • No need to have the image subdor in the csratchdir when USB.
  • umount the USB partition before attempting to use it
  • Remove useless copy from mindi to mondo at end of USB handling

(risky merge, we are raising the limits of 2 diverging branches. The status of stable is not completely sure as such. Will need lots of tests, but it's not yet done :-()
(merge -r1692:1769 $SVN_M/branches/2.2.5)

File size: 21.7 KB
Line 
1/* vi: set sw=4 ts=4: */
2/*
3 * wget - retrieve a file using HTTP or FTP
4 *
5 * Chip Rosenthal Covad Communications <chip@laserlink.net>
6 *
7 */
8
9/* We want libc to give us xxx64 functions also */
10/* http://www.unix.org/version2/whatsnew/lfs20mar.html */
11//#define _LARGEFILE64_SOURCE 1
12
13#include <getopt.h> /* for struct option */
14#include "libbb.h"
15
16struct host_info {
17 // May be used if we ever will want to free() all xstrdup()s...
18 /* char *allocated; */
19 char *host;
20 int port;
21 char *path;
22 int is_ftp;
23 char *user;
24};
25
26static void parse_url(char *url, struct host_info *h);
27static FILE *open_socket(len_and_sockaddr *lsa);
28static char *gethdr(char *buf, size_t bufsiz, FILE *fp, int *istrunc);
29static int ftpcmd(const char *s1, const char *s2, FILE *fp, char *buf);
30
31/* Globals (can be accessed from signal handlers */
32static off_t content_len; /* Content-length of the file */
33static off_t beg_range; /* Range at which continue begins */
34#if ENABLE_FEATURE_WGET_STATUSBAR
35static off_t transferred; /* Number of bytes transferred so far */
36#endif
37static bool chunked; /* chunked transfer encoding */
38#if ENABLE_FEATURE_WGET_STATUSBAR
39static void progressmeter(int flag);
40static const char *curfile; /* Name of current file being transferred */
41enum {
42 STALLTIME = 5 /* Seconds when xfer considered "stalled" */
43};
44#else
45static ALWAYS_INLINE void progressmeter(int flag) {}
46#endif
47
48/* Read NMEMB bytes into PTR from STREAM. Returns the number of bytes read,
49 * and a short count if an eof or non-interrupt error is encountered. */
50static size_t safe_fread(void *ptr, size_t nmemb, FILE *stream)
51{
52 size_t ret;
53 char *p = (char*)ptr;
54
55 do {
56 clearerr(stream);
57 ret = fread(p, 1, nmemb, stream);
58 p += ret;
59 nmemb -= ret;
60 } while (nmemb && ferror(stream) && errno == EINTR);
61
62 return p - (char*)ptr;
63}
64
65/* Read a line or SIZE-1 bytes into S, whichever is less, from STREAM.
66 * Returns S, or NULL if an eof or non-interrupt error is encountered. */
67static char *safe_fgets(char *s, int size, FILE *stream)
68{
69 char *ret;
70
71 do {
72 clearerr(stream);
73 ret = fgets(s, size, stream);
74 } while (ret == NULL && ferror(stream) && errno == EINTR);
75
76 return ret;
77}
78
79#if ENABLE_FEATURE_WGET_AUTHENTICATION
80/* Base64-encode character string. buf is assumed to be char buf[512]. */
81static char *base64enc_512(char buf[512], const char *str)
82{
83 unsigned len = strlen(str);
84 if (len > 512/4*3 - 10) /* paranoia */
85 len = 512/4*3 - 10;
86 bb_uuencode(buf, str, len, bb_uuenc_tbl_base64);
87 return buf;
88}
89#endif
90
91int wget_main(int argc, char **argv);
92int wget_main(int argc, char **argv)
93{
94 char buf[512];
95 struct host_info server, target;
96 len_and_sockaddr *lsa;
97 int n, status;
98 int port;
99 int try = 5;
100 unsigned opt;
101 char *str;
102 char *proxy = 0;
103 char *dir_prefix = NULL;
104#if ENABLE_FEATURE_WGET_LONG_OPTIONS
105 char *extra_headers = NULL;
106 llist_t *headers_llist = NULL;
107#endif
108
109 FILE *sfp = NULL; /* socket to web/ftp server */
110 FILE *dfp = NULL; /* socket to ftp server (data) */
111 char *fname_out = NULL; /* where to direct output (-O) */
112 bool got_clen = 0; /* got content-length: from server */
113 int output_fd = -1;
114 bool use_proxy = 1; /* Use proxies if env vars are set */
115 const char *proxy_flag = "on"; /* Use proxies if env vars are set */
116 const char *user_agent = "Wget";/* "User-Agent" header field */
117 static const char keywords[] ALIGN1 =
118 "content-length\0""transfer-encoding\0""chunked\0""location\0";
119 enum {
120 KEY_content_length = 1, KEY_transfer_encoding, KEY_chunked, KEY_location
121 };
122 enum {
123 WGET_OPT_CONTINUE = 0x1,
124 WGET_OPT_SPIDER = 0x2,
125 WGET_OPT_QUIET = 0x4,
126 WGET_OPT_OUTNAME = 0x8,
127 WGET_OPT_PREFIX = 0x10,
128 WGET_OPT_PROXY = 0x20,
129 WGET_OPT_USER_AGENT = 0x40,
130 WGET_OPT_PASSIVE = 0x80,
131 WGET_OPT_HEADER = 0x100,
132 };
133#if ENABLE_FEATURE_WGET_LONG_OPTIONS
134 static const char wget_longopts[] ALIGN1 =
135 /* name, has_arg, val */
136 "continue\0" No_argument "c"
137 "spider\0" No_argument "s"
138 "quiet\0" No_argument "q"
139 "output-document\0" Required_argument "O"
140 "directory-prefix\0" Required_argument "P"
141 "proxy\0" Required_argument "Y"
142 "user-agent\0" Required_argument "U"
143 "passive-ftp\0" No_argument "\xff"
144 "header\0" Required_argument "\xfe"
145 ;
146 applet_long_options = wget_longopts;
147#endif
148 /* server.allocated = target.allocated = NULL; */
149 opt_complementary = "-1" USE_FEATURE_WGET_LONG_OPTIONS(":\xfe::");
150 opt = getopt32(argv, "csqO:P:Y:U:",
151 &fname_out, &dir_prefix,
152 &proxy_flag, &user_agent
153 USE_FEATURE_WGET_LONG_OPTIONS(, &headers_llist)
154 );
155 if (strcmp(proxy_flag, "off") == 0) {
156 /* Use the proxy if necessary */
157 use_proxy = 0;
158 }
159#if ENABLE_FEATURE_WGET_LONG_OPTIONS
160 if (headers_llist) {
161 int size = 1;
162 char *cp;
163 llist_t *ll = headers_llist;
164 while (ll) {
165 size += strlen(ll->data) + 2;
166 ll = ll->link;
167 }
168 extra_headers = cp = xmalloc(size);
169 while (headers_llist) {
170 cp += sprintf(cp, "%s\r\n", headers_llist->data);
171 headers_llist = headers_llist->link;
172 }
173 }
174#endif
175
176 parse_url(argv[optind], &target);
177 server.host = target.host;
178 server.port = target.port;
179
180 /* Use the proxy if necessary */
181 if (use_proxy) {
182 proxy = getenv(target.is_ftp ? "ftp_proxy" : "http_proxy");
183 if (proxy && *proxy) {
184 parse_url(proxy, &server);
185 } else {
186 use_proxy = 0;
187 }
188 }
189
190 /* Guess an output filename */
191 if (!fname_out) {
192 // Dirty hack. Needed because bb_get_last_path_component
193 // will destroy trailing / by storing '\0' in last byte!
194 if (!last_char_is(target.path, '/')) {
195 fname_out = bb_get_last_path_component(target.path);
196#if ENABLE_FEATURE_WGET_STATUSBAR
197 curfile = fname_out;
198#endif
199 }
200 if (!fname_out || !fname_out[0]) {
201 /* bb_get_last_path_component writes
202 * to last '/' only. We don't have one here... */
203 fname_out = (char*)"index.html";
204#if ENABLE_FEATURE_WGET_STATUSBAR
205 curfile = fname_out;
206#endif
207 }
208 if (dir_prefix != NULL)
209 fname_out = concat_path_file(dir_prefix, fname_out);
210#if ENABLE_FEATURE_WGET_STATUSBAR
211 } else {
212 curfile = bb_get_last_path_component(fname_out);
213#endif
214 }
215 /* Impossible?
216 if ((opt & WGET_OPT_CONTINUE) && !fname_out)
217 bb_error_msg_and_die("cannot specify continue (-c) without a filename (-O)"); */
218
219 /* Determine where to start transfer */
220 if (LONE_DASH(fname_out)) {
221 output_fd = 1;
222 opt &= ~WGET_OPT_CONTINUE;
223 }
224 if (opt & WGET_OPT_CONTINUE) {
225 output_fd = open(fname_out, O_WRONLY);
226 if (output_fd >= 0) {
227 beg_range = xlseek(output_fd, 0, SEEK_END);
228 }
229 /* File doesn't exist. We do not create file here yet.
230 We are not sure it exists on remove side */
231 }
232
233 /* We want to do exactly _one_ DNS lookup, since some
234 * sites (i.e. ftp.us.debian.org) use round-robin DNS
235 * and we want to connect to only one IP... */
236 lsa = xhost2sockaddr(server.host, server.port);
237 if (!(opt & WGET_OPT_QUIET)) {
238 fprintf(stderr, "Connecting to %s (%s)\n", server.host,
239 xmalloc_sockaddr2dotted(&lsa->sa));
240 /* We leak result of xmalloc_sockaddr2dotted */
241 }
242
243 if (use_proxy || !target.is_ftp) {
244 /*
245 * HTTP session
246 */
247 do {
248 got_clen = chunked = 0;
249
250 if (!--try)
251 bb_error_msg_and_die("too many redirections");
252
253 /* Open socket to http server */
254 if (sfp) fclose(sfp);
255 sfp = open_socket(lsa);
256
257 /* Send HTTP request. */
258 if (use_proxy) {
259 fprintf(sfp, "GET %stp://%s/%s HTTP/1.1\r\n",
260 target.is_ftp ? "f" : "ht", target.host,
261 target.path);
262 } else {
263 fprintf(sfp, "GET /%s HTTP/1.1\r\n", target.path);
264 }
265
266 fprintf(sfp, "Host: %s\r\nUser-Agent: %s\r\n",
267 target.host, user_agent);
268
269#if ENABLE_FEATURE_WGET_AUTHENTICATION
270 if (target.user) {
271 fprintf(sfp, "Proxy-Authorization: Basic %s\r\n"+6,
272 base64enc_512(buf, target.user));
273 }
274 if (use_proxy && server.user) {
275 fprintf(sfp, "Proxy-Authorization: Basic %s\r\n",
276 base64enc_512(buf, server.user));
277 }
278#endif
279
280 if (beg_range)
281 fprintf(sfp, "Range: bytes=%"OFF_FMT"d-\r\n", beg_range);
282#if ENABLE_FEATURE_WGET_LONG_OPTIONS
283 if (extra_headers)
284 fputs(extra_headers, sfp);
285#endif
286 fprintf(sfp, "Connection: close\r\n\r\n");
287
288 /*
289 * Retrieve HTTP response line and check for "200" status code.
290 */
291 read_response:
292 if (fgets(buf, sizeof(buf), sfp) == NULL)
293 bb_error_msg_and_die("no response from server");
294
295 str = buf;
296 str = skip_non_whitespace(str);
297 str = skip_whitespace(str);
298 // FIXME: no error check
299 // xatou wouldn't work: "200 OK"
300 status = atoi(str);
301 switch (status) {
302 case 0:
303 case 100:
304 while (gethdr(buf, sizeof(buf), sfp, &n) != NULL)
305 /* eat all remaining headers */;
306 goto read_response;
307 case 200:
308 break;
309 case 300: /* redirection */
310 case 301:
311 case 302:
312 case 303:
313 break;
314 case 206:
315 if (beg_range)
316 break;
317 /*FALLTHRU*/
318 default:
319 /* Show first line only and kill any ESC tricks */
320 buf[strcspn(buf, "\n\r\x1b")] = '\0';
321 bb_error_msg_and_die("server returned error: %s", buf);
322 }
323
324 /*
325 * Retrieve HTTP headers.
326 */
327 while ((str = gethdr(buf, sizeof(buf), sfp, &n)) != NULL) {
328 /* gethdr did already convert the "FOO:" string to lowercase */
329 smalluint key = index_in_strings(keywords, *&buf) + 1;
330 if (key == KEY_content_length) {
331 content_len = BB_STRTOOFF(str, NULL, 10);
332 if (errno || content_len < 0) {
333 bb_error_msg_and_die("content-length %s is garbage", str);
334 }
335 got_clen = 1;
336 continue;
337 }
338 if (key == KEY_transfer_encoding) {
339 if (index_in_strings(keywords, str_tolower(str)) + 1 != KEY_chunked)
340 bb_error_msg_and_die("server wants to do %s transfer encoding", str);
341 chunked = got_clen = 1;
342 }
343 if (key == KEY_location) {
344 if (str[0] == '/')
345 /* free(target.allocated); */
346 target.path = /* target.allocated = */ xstrdup(str+1);
347 else {
348 parse_url(str, &target);
349 if (use_proxy == 0) {
350 server.host = target.host;
351 server.port = target.port;
352 }
353 free(lsa);
354 lsa = xhost2sockaddr(server.host, server.port);
355 break;
356 }
357 }
358 }
359 } while (status >= 300);
360
361 dfp = sfp;
362
363 } else {
364
365 /*
366 * FTP session
367 */
368 if (!target.user)
369 target.user = xstrdup("anonymous:busybox@");
370
371 sfp = open_socket(lsa);
372 if (ftpcmd(NULL, NULL, sfp, buf) != 220)
373 bb_error_msg_and_die("%s", buf+4);
374
375 /*
376 * Splitting username:password pair,
377 * trying to log in
378 */
379 str = strchr(target.user, ':');
380 if (str)
381 *(str++) = '\0';
382 switch (ftpcmd("USER ", target.user, sfp, buf)) {
383 case 230:
384 break;
385 case 331:
386 if (ftpcmd("PASS ", str, sfp, buf) == 230)
387 break;
388 /* FALLTHRU (failed login) */
389 default:
390 bb_error_msg_and_die("ftp login: %s", buf+4);
391 }
392
393 ftpcmd("TYPE I", NULL, sfp, buf);
394
395 /*
396 * Querying file size
397 */
398 if (ftpcmd("SIZE ", target.path, sfp, buf) == 213) {
399 content_len = BB_STRTOOFF(buf+4, NULL, 10);
400 if (errno || content_len < 0) {
401 bb_error_msg_and_die("SIZE value is garbage");
402 }
403 got_clen = 1;
404 }
405
406 /*
407 * Entering passive mode
408 */
409 if (ftpcmd("PASV", NULL, sfp, buf) != 227) {
410 pasv_error:
411 bb_error_msg_and_die("bad response to %s: %s", "PASV", buf);
412 }
413 // Response is "227 garbageN1,N2,N3,N4,P1,P2[)garbage]
414 // Server's IP is N1.N2.N3.N4 (we ignore it)
415 // Server's port for data connection is P1*256+P2
416 str = strrchr(buf, ')');
417 if (str) str[0] = '\0';
418 str = strrchr(buf, ',');
419 if (!str) goto pasv_error;
420 port = xatou_range(str+1, 0, 255);
421 *str = '\0';
422 str = strrchr(buf, ',');
423 if (!str) goto pasv_error;
424 port += xatou_range(str+1, 0, 255) * 256;
425 set_nport(lsa, htons(port));
426 dfp = open_socket(lsa);
427
428 if (beg_range) {
429 sprintf(buf, "REST %"OFF_FMT"d", beg_range);
430 if (ftpcmd(buf, NULL, sfp, buf) == 350)
431 content_len -= beg_range;
432 }
433
434 if (ftpcmd("RETR ", target.path, sfp, buf) > 150)
435 bb_error_msg_and_die("bad response to RETR: %s", buf);
436 }
437 if (opt & WGET_OPT_SPIDER) {
438 if (ENABLE_FEATURE_CLEAN_UP)
439 fclose(sfp);
440 goto done;
441 }
442
443 /*
444 * Retrieve file
445 */
446 if (chunked) {
447 fgets(buf, sizeof(buf), dfp);
448 content_len = STRTOOFF(buf, NULL, 16);
449 /* FIXME: error check?? */
450 }
451
452 /* Do it before progressmeter (want to have nice error message) */
453 if (output_fd < 0)
454 output_fd = xopen(fname_out,
455 O_WRONLY|O_CREAT|O_EXCL|O_TRUNC);
456
457 if (!(opt & WGET_OPT_QUIET))
458 progressmeter(-1);
459
460 do {
461 while (content_len > 0 || !got_clen) {
462 unsigned rdsz = sizeof(buf);
463 if (content_len < sizeof(buf) && (chunked || got_clen))
464 rdsz = (unsigned)content_len;
465 n = safe_fread(buf, rdsz, dfp);
466 if (n <= 0)
467 break;
468 if (full_write(output_fd, buf, n) != n) {
469 bb_perror_msg_and_die(bb_msg_write_error);
470 }
471#if ENABLE_FEATURE_WGET_STATUSBAR
472 transferred += n;
473#endif
474 if (got_clen) {
475 content_len -= n;
476 }
477 }
478
479 if (chunked) {
480 safe_fgets(buf, sizeof(buf), dfp); /* This is a newline */
481 safe_fgets(buf, sizeof(buf), dfp);
482 content_len = STRTOOFF(buf, NULL, 16);
483 /* FIXME: error check? */
484 if (content_len == 0) {
485 chunked = 0; /* all done! */
486 }
487 }
488
489 if (n == 0 && ferror(dfp)) {
490 bb_perror_msg_and_die(bb_msg_read_error);
491 }
492 } while (chunked);
493
494 if (!(opt & WGET_OPT_QUIET))
495 progressmeter(1);
496
497 if ((use_proxy == 0) && target.is_ftp) {
498 fclose(dfp);
499 if (ftpcmd(NULL, NULL, sfp, buf) != 226)
500 bb_error_msg_and_die("ftp error: %s", buf+4);
501 ftpcmd("QUIT", NULL, sfp, buf);
502 }
503done:
504 exit(EXIT_SUCCESS);
505}
506
507
508static void parse_url(char *src_url, struct host_info *h)
509{
510 char *url, *p, *sp;
511
512 /* h->allocated = */ url = xstrdup(src_url);
513
514 if (strncmp(url, "http://", 7) == 0) {
515 h->port = bb_lookup_port("http", "tcp", 80);
516 h->host = url + 7;
517 h->is_ftp = 0;
518 } else if (strncmp(url, "ftp://", 6) == 0) {
519 h->port = bb_lookup_port("ftp", "tcp", 21);
520 h->host = url + 6;
521 h->is_ftp = 1;
522 } else
523 bb_error_msg_and_die("not an http or ftp url: %s", url);
524
525 // FYI:
526 // "Real" wget 'http://busybox.net?var=a/b' sends this request:
527 // 'GET /?var=a/b HTTP 1.0'
528 // and saves 'index.html?var=a%2Fb' (we save 'b')
529 // wget 'http://busybox.net?login=john@doe':
530 // request: 'GET /?login=john@doe HTTP/1.0'
531 // saves: 'index.html?login=john@doe' (we save '?login=john@doe')
532 // wget 'http://busybox.net#test/test':
533 // request: 'GET / HTTP/1.0'
534 // saves: 'index.html' (we save 'test')
535 //
536 // We also don't add unique .N suffix if file exists...
537 sp = strchr(h->host, '/');
538 p = strchr(h->host, '?'); if (!sp || (p && sp > p)) sp = p;
539 p = strchr(h->host, '#'); if (!sp || (p && sp > p)) sp = p;
540 if (!sp) {
541 /* must be writable because of bb_get_last_path_component() */
542 static char nullstr[] ALIGN1 = "";
543 h->path = nullstr;
544 } else if (*sp == '/') {
545 *sp = '\0';
546 h->path = sp + 1;
547 } else { // '#' or '?'
548 // http://busybox.net?login=john@doe is a valid URL
549 // memmove converts to:
550 // http:/busybox.nett?login=john@doe...
551 memmove(h->host-1, h->host, sp - h->host);
552 h->host--;
553 sp[-1] = '\0';
554 h->path = sp;
555 }
556
557 sp = strrchr(h->host, '@');
558 h->user = NULL;
559 if (sp != NULL) {
560 h->user = h->host;
561 *sp = '\0';
562 h->host = sp + 1;
563 }
564
565 sp = h->host;
566}
567
568
569static FILE *open_socket(len_and_sockaddr *lsa)
570{
571 FILE *fp;
572
573 /* glibc 2.4 seems to try seeking on it - ??! */
574 /* hopefully it understands what ESPIPE means... */
575 fp = fdopen(xconnect_stream(lsa), "r+");
576 if (fp == NULL)
577 bb_perror_msg_and_die("fdopen");
578
579 return fp;
580}
581
582
583static char *gethdr(char *buf, size_t bufsiz, FILE *fp, int *istrunc)
584{
585 char *s, *hdrval;
586 int c;
587
588 *istrunc = 0;
589
590 /* retrieve header line */
591 if (fgets(buf, bufsiz, fp) == NULL)
592 return NULL;
593
594 /* see if we are at the end of the headers */
595 for (s = buf; *s == '\r'; ++s)
596 ;
597 if (s[0] == '\n')
598 return NULL;
599
600 /* convert the header name to lower case */
601 for (s = buf; isalnum(*s) || *s == '-'; ++s)
602 *s = tolower(*s);
603
604 /* verify we are at the end of the header name */
605 if (*s != ':')
606 bb_error_msg_and_die("bad header line: %s", buf);
607
608 /* locate the start of the header value */
609 for (*s++ = '\0'; *s == ' ' || *s == '\t'; ++s)
610 ;
611 hdrval = s;
612
613 /* locate the end of header */
614 while (*s != '\0' && *s != '\r' && *s != '\n')
615 ++s;
616
617 /* end of header found */
618 if (*s != '\0') {
619 *s = '\0';
620 return hdrval;
621 }
622
623 /* Rats! The buffer isn't big enough to hold the entire header value. */
624 while (c = getc(fp), c != EOF && c != '\n')
625 ;
626 *istrunc = 1;
627 return hdrval;
628}
629
630static int ftpcmd(const char *s1, const char *s2, FILE *fp, char *buf)
631{
632 int result;
633 if (s1) {
634 if (!s2) s2 = "";
635 fprintf(fp, "%s%s\r\n", s1, s2);
636 fflush(fp);
637 }
638
639 do {
640 char *buf_ptr;
641
642 if (fgets(buf, 510, fp) == NULL) {
643 bb_perror_msg_and_die("error getting response");
644 }
645 buf_ptr = strstr(buf, "\r\n");
646 if (buf_ptr) {
647 *buf_ptr = '\0';
648 }
649 } while (!isdigit(buf[0]) || buf[3] != ' ');
650
651 buf[3] = '\0';
652 result = xatoi_u(buf);
653 buf[3] = ' ';
654 return result;
655}
656
657#if ENABLE_FEATURE_WGET_STATUSBAR
658/* Stuff below is from BSD rcp util.c, as added to openshh.
659 * Original copyright notice is retained at the end of this file.
660 */
661static int
662getttywidth(void)
663{
664 int width;
665 get_terminal_width_height(0, &width, NULL);
666 return width;
667}
668
669static void
670updateprogressmeter(int ignore)
671{
672 int save_errno = errno;
673
674 progressmeter(0);
675 errno = save_errno;
676}
677
678static void alarmtimer(int iwait)
679{
680 struct itimerval itv;
681
682 itv.it_value.tv_sec = iwait;
683 itv.it_value.tv_usec = 0;
684 itv.it_interval = itv.it_value;
685 setitimer(ITIMER_REAL, &itv, NULL);
686}
687
688static void
689progressmeter(int flag)
690{
691 static unsigned lastupdate_sec;
692 static unsigned start_sec;
693 static off_t lastsize, totalsize;
694
695 off_t abbrevsize;
696 unsigned since_last_update, elapsed;
697 unsigned ratio;
698 int barlength, i;
699
700 if (flag == -1) { /* first call to progressmeter */
701 start_sec = monotonic_sec();
702 lastupdate_sec = start_sec;
703 lastsize = 0;
704 totalsize = content_len + beg_range; /* as content_len changes.. */
705 }
706
707 ratio = 100;
708 if (totalsize != 0 && !chunked) {
709 /* long long helps to have it working even if !LFS */
710 ratio = (unsigned) (100ULL * (transferred+beg_range) / totalsize);
711 if (ratio > 100) ratio = 100;
712 }
713
714 fprintf(stderr, "\r%-20.20s%4d%% ", curfile, ratio);
715
716 barlength = getttywidth() - 49;
717 if (barlength > 0) {
718 /* god bless gcc for variable arrays :) */
719 i = barlength * ratio / 100;
720 {
721 char buf[i+1];
722 memset(buf, '*', i);
723 buf[i] = '\0';
724 fprintf(stderr, "|%s%*s|", buf, barlength - i, "");
725 }
726 }
727 i = 0;
728 abbrevsize = transferred + beg_range;
729 while (abbrevsize >= 100000) {
730 i++;
731 abbrevsize >>= 10;
732 }
733 /* see http://en.wikipedia.org/wiki/Tera */
734 fprintf(stderr, "%6d%c ", (int)abbrevsize, " kMGTPEZY"[i]);
735
736// Nuts! Ain't it easier to update progress meter ONLY when we transferred++?
737// FIXME: get rid of alarmtimer + updateprogressmeter mess
738
739 elapsed = monotonic_sec();
740 since_last_update = elapsed - lastupdate_sec;
741 if (transferred > lastsize) {
742 lastupdate_sec = elapsed;
743 lastsize = transferred;
744 if (since_last_update >= STALLTIME) {
745 /* We "cut off" these seconds from elapsed time
746 * by adjusting start time */
747 start_sec += since_last_update;
748 }
749 since_last_update = 0; /* we are un-stalled now */
750 }
751 elapsed -= start_sec; /* now it's "elapsed since start" */
752
753 if (since_last_update >= STALLTIME) {
754 fprintf(stderr, " - stalled -");
755 } else {
756 off_t to_download = totalsize - beg_range;
757 if (transferred <= 0 || (int)elapsed <= 0 || transferred > to_download || chunked) {
758 fprintf(stderr, "--:--:-- ETA");
759 } else {
760 /* to_download / (transferred/elapsed) - elapsed: */
761 int eta = (int) ((unsigned long long)to_download*elapsed/transferred - elapsed);
762 /* (long long helps to have working ETA even if !LFS) */
763 i = eta % 3600;
764 fprintf(stderr, "%02d:%02d:%02d ETA", eta / 3600, i / 60, i % 60);
765 }
766 }
767
768 if (flag == -1) { /* first call to progressmeter */
769 struct sigaction sa;
770 sa.sa_handler = updateprogressmeter;
771 sigemptyset(&sa.sa_mask);
772 sa.sa_flags = SA_RESTART;
773 sigaction(SIGALRM, &sa, NULL);
774 alarmtimer(1);
775 } else if (flag == 1) { /* last call to progressmeter */
776 alarmtimer(0);
777 transferred = 0;
778 putc('\n', stderr);
779 }
780}
781#endif /* FEATURE_WGET_STATUSBAR */
782
783/* Original copyright notice which applies to the CONFIG_FEATURE_WGET_STATUSBAR stuff,
784 * much of which was blatantly stolen from openssh. */
785
786/*-
787 * Copyright (c) 1992, 1993
788 * The Regents of the University of California. All rights reserved.
789 *
790 * Redistribution and use in source and binary forms, with or without
791 * modification, are permitted provided that the following conditions
792 * are met:
793 * 1. Redistributions of source code must retain the above copyright
794 * notice, this list of conditions and the following disclaimer.
795 * 2. Redistributions in binary form must reproduce the above copyright
796 * notice, this list of conditions and the following disclaimer in the
797 * documentation and/or other materials provided with the distribution.
798 *
799 * 3. <BSD Advertising Clause omitted per the July 22, 1999 licensing change
800 * ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change>
801 *
802 * 4. Neither the name of the University nor the names of its contributors
803 * may be used to endorse or promote products derived from this software
804 * without specific prior written permission.
805 *
806 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
807 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
808 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
809 * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
810 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
811 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
812 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
813 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
814 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
815 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
816 * SUCH DAMAGE.
817 *
818 */
Note: See TracBrowser for help on using the repository browser.