Search Engine Spider-Friendly JavaScript Content
If you have syndicated content delivered with JavaScript,
the content might not be indexed by the search engines.
Syndicated content, in this article, means HTML content
(text, images, forms, and so forth) delivered from a remote
site and published on your page with JavaScript code similar
to:
<script
language="JavaScript"
src="http://example.com/file.js">
</script>
If you syndicate the Possibilities articles, you're familiar
with the above. The software used to prepare and deliver
JavaScript syndicated content is Master Syndicator V4 from
/msyn or Master Syndication Gateway V2
from /gateway
It make the syndicated content search engine spider-friendly,
use an SSI tag to insert the content into your web page. The
SSI tag runs a CGI program that fetches and delivers the
content.
The CGI Program
Here is the CGI program that will fetch the syndicated
content and send it to your web page:
#!/usr/bin/perl
my $JavaScriptURL = 'http://example.com/file.js';
use strict;
use LWP::Simple;
my $content = get $JavaScriptURL;
$content =~ s/^\s*<!--//s;
$content =~ s!(?://)?\s*-->\s*$!!s;
my @content = split /[\r\n]+/,$content;
for(@content)
{
unless(/^document/) { $_ = ''; next; }
$_ =~ s/^document\.writeln\(\'//;
$_ =~ s/'\)\;$//;
$_ =~ s/' ?\+ ?'//g;
$_ =~ s/\\'/'/g;
$_ =~ s/\\\\/\\/g;
}
$content = join "\n",@content;
print "Content-type: text/html\n\n$content";
# end of script
Edit and save the above program using a plain text word
processor like NotePad or TextWrangler.
-
The first line must point to the location of Perl
on your server.
-
On the second line, replace the URL between the
apostrophes with the URL of the syndicated content.
This would be the URL found in the src="_________"
attribute of the syndication <script...> tag. (To
syndicate WillMaster Possibilities articles, use the
/library/c/wmp.js URL.)
Upload the program to your server as ASCII/plain text, to a
directory that can run CGI scripts. Give it 755 permissions.
It requires the Perl module LWP::Simple. If you are unsure
whether or not your server has the module installed, ask
your hosting company or use Master Pre-Installation Tester
from /mpits
The SSI Tag
Here is the SSI tag that will run the CGI program:
<!--#exec cgi="/cgi-bin/programfile.cgi"-->
Replace "/cgi-bin/programfile.cgi" with the location and
name of the CGI program on your server.
Use the SSI tag instead of the JavaScript syndication code.
On many servers, pages with SSI need to have file names
with the .shtml extension. Your server may be different.
Contact your hosting company if in doubt.
When you've installed the CGI program and use the SSI tag
in lieu of the JavaScript syndication code, search engine
spiders can index your syndicated content as if it was
static content.
Question:
Did you find this article interesting and understandable? How can it be improved?
Your response is anonymous.
When done typing, click anywhere outside the box. [more info]
Will Bontrager
©2005 Bontrager Connection, LLC
Please note:
Articles on this website are presented "as is". However -
If you have a question about a CGI script, HTML, CSS, PHP, or JavaScript
Ask one of our Experts and you'll have your answer!
Click here for details.
