
Solid, safe website tools.
Whatever your need, Will Bontrager builds powerful software solutions.
If you don't find the answer to your question in these archives, ask your CGI question at the Current Master Series CGI Forum.
| Author | Message |
|---|---|
| 1Dec04 Phil Tanny |
Subject: Browser timeouts Howdy Will, hope some friendly tumbleweeds are blowing your way today. Can you offer some general guidance on how to deal with situations when the browser times out before a perl script is finished? As example, I have a routine that grabs a web page and finds some data. It works fine if I process a few pages, but the browser times out if the script is fed too many pages to look for. Or, mailing to a list. Works great if the list is small, browser times out before mailing to a large list is complete. I know I can trigger the scripts via telnet, and I've read suggestions to set up a cron that constantly checks a "job list" and triggers scripts when new jobs are present. Both of these solutions seem really clunky and seem to complicate the job considerably. Is it possible to launch a long running script from a web page form, have it print an "in progress" report to the browser, and then email me when the script is complete? Or does printing anything to the browser always end the script? OK, you get the idea. I'm sure this must be a common question from newbie perl writers. Thanks! Phil |
| 2Dec04 Will [Email] ![]() |
In response to: Browser timeouts That could be a tough one, Phil. Put $|=1;(that's a vertical bar character following the $ sign) on a line above the print statements for content going to the browser. That will autoflush the content whenever a newline character is encountered. However, just because it flushes the pipe to the browser, doesn't mean a server or the browser won't cache it. I'm sure there's a better solution, but the few times I've needed to keep a browser alert, I've send a 1025 following every print block. Like print "\n" x 1025; That could quickly become a very large page if it's done a lot. Speeding up the scripts would help, but somethimes that can't be done, or not enough to make the difference. Can the process be accomplished in two phases, maybe? If yes, the first phase could draw a web page with hidden fields containing whatever information the second phase needs. The user can then be instructed to "click this button to complete the process" or something like that. Sorry I don't have a "this will work" answer. |
| 3Dec04 Phil Tanny |
In response to: Browser timeouts Hi Will, Thanks for your input, much appreciated! Maybe this will help focus the question a bit. I don't actually care whether the browser times out, as these are routines that only I use. What I care about is whether the routine finishes the job. Does a routine die once the browser times out? If I could input data and launch the process with a browser, and then get a confirmation email when the process is complete, that would be fine. It would't matter if the browser died in the middle, so long as the routine contined sending mail, checking web pages, or whatever it is supposed to do. Does that solve the problem, or does the browser time out signal the end of processing? Thanks again! |
| 3Dec04 Will |
In response to: Browser timeouts Yes, the browser timing out would signal the CGI program that it's no longer needed. CGI is a gateway, and closing something on one side, the browser or the program, closes the gate. An idea, if your server can relay email to a script, set up an email address to nudge the program you want to run. You would bypass the CGI protocol this way. When you're ready to run the program, send an email to that special address. The script can email you back when its done. |
| 3Dec04 Phil Tanny |
In response to: Browser timeouts OK, thanks Will. I see now that I can trigger a script via cron, telnet or email, thus bypassing the browser limitation. That solves one issue. I use the browser to tell the script what to do as well. You know, send this message, to this list etc. So I guess I can use my web interface to print this input to a file. Then recode the script to look for the instructions and data it needs in this file. Then turn the script on by one of the above methods. Does this sound sensible to you? Know a better way? Darn, 2 months of work on scripts that work great, until the browser kills them. Oh well, back to the drawing board. :-( Thanks as always Will. |
| 3Dec04 Will [Email] ![]() |
In response to: Browser timeouts There's no need to rewrite lots of stuff. Instead of using a browser to make the instruction file and then sending an email to nudge the script, the email itself might hold the instructions. Put the instructions into the body so they can be parsed, maybe something like File: something.txt NewFile: other.txt Start: 'A' End: 'E' or whatever the instructions need to be. Those lines can then be parsed with something like my %Instructions = ();
for(split /\n/,$Email)
{
next unless /:/;
my($name,$value) = split /:/,$_,2;
$Instructions{$name} = $value;
}The script to grab the email and discard the header lines could be something like #!/usr/bin/perl
use strict;
# Grab the email relayed to the script
my $Email = join '',<STDIN>;
# Normalize to Unix line endings
if($Email =~ /\n/) { $Email =~ s/\r//g; }
else { $Email =~ s/\r/\n/g; }
# Discard the email header lines
# (blank line separates header from body)
(undef,$Email) = split /\n\n/,$Email,2;
# $Email now contains the body of the email.So that's one way. If you prefer to use a form, maybe so you don't have to remember how the email is supposed to be formatted, you could use a form processor like Master Form V3 to generate the email and send it to the script's email address. It's late, and I didn't test the above code, but I think you see the idea. |
| 4Dec04 Phil Tanny |
In response to: Browser timeouts Will, I like your idea of triggering a script with an email. I could send the email from my admin interface, using the same start button I've already got in place. I get that I need to set up a special email account for this purpose. But how do I direct the email from that account to the script? For now I just want to send an email that triggers the script. I've done some google searching, and suspect I need to add an entry to some file on the server, something to do with procmail? I manage my web accounts with Cpanel, which has a million neato options, but I don't see anything like this. Can you help me frame a question to my host? Thanks! |
| 4Dec04 Will |
In response to: Browser timeouts Hi Phil, If you can't do it through cpanel (I don't know cpanel very well), check out the Master Autoresponder Bank manual "Requesting an Auto-Response By Email" section. It has general instructions for several different methods of getting an incoming email to a script sendmail's "aliases" file, procmail, and the Unix ".forward" file. If it can be done with cpanel, it would probably be in the section where you forward email, and the forwarded address might be something like |/path/to/perl /path/to/script.cgi(The vertical bar at the beginning might or might not be required.) An example email for your hosting company: Hello People, I'm installing a Perl script that will process email sent to a certain email address. The server will need to relay/pipe certain email to the script. Would that be done with Unix the ".forward" file, the sendmail "aliases" file, the procmail "procmailrc" file, or some other way? |
| 5Dec04 Phil Tanny |
In response to: Browser timeouts It's working now Will, thanks! Cpanel makes it really easy to do the forward email to script method. And your coding that grabs the body of the email works. This is a very useful thing to learn, now I'm glad I ran in to the browser timeout issue. :-) Much appreciated. I just bought a script from your site to say thanks in a more tangible manner. Hey, the charity donation aspect is cool, good for you. |
| 5Dec04 Will [Email] ![]() |
In response to: Browser timeouts I'm glad the problem resulted in benefits beyond the original question. Like "they" say, there is a reason for everything :) (Who the heck is "they," anyway? I want to talk to those folks about a few things they simply got wrong.) Thank you for mentioning the charity drive. Yes, 20% off on the CGI programs and half of the proceeds go to charity. This is the second year we've done this. Last year a nice check went to charity. We (and they) appreciate any word of mouth promotion the drive gets. |