Search Terms Extractor

Would you like to know the search terms visitors use to find your page?

The Search Terms Extractor software makes that possible.

Search terms visitors use to find your web pages may be used in any or all of these ways:

Printed on the page itself.
Logged to a file on the server,
Emailed to you.

I'll show you how to do it, and provide the software to do it with.

Printing Visitors' Search Terms On the Web Page

Some months ago, I saw a web page with the search term I used to find it printed right on the page itself. It was something like, "Looking for [search term]?", followed by an ad for a book related to the subject of the web page.

I thought, "How clever. I'll have to make one of those."

The Search Terms Extractor implementation instructions show how to print the search terms on a web page. The identity of the search engine used to find the page can also be printed.

Logging Search Terms to a File

When search terms are logged to a file, they can later be retrieved and processed.

Search word and phrase frequency can be determined. Search words and phrases can be listed, sorted, sifted, and printed. Some might be appropriate for use as key words in other advertising.

The implementation instructions show how to log the search terms.

When the search terms are logged, any or all of the following can also be logged:

The date and time of the visitor's arrival.
The identity of the search engine used.
The web page the visitor arrived at.

The log file is formatted as a CSV file.

Email Notification When Visitor Arrives From Search Engine

When you choose to be notified whenever someone finds your web page in a search engine, the email contains this information:

The search word or phrase.
The identity of the search engine used.
The web page the visitor arrived at.

The implementation instructions show how to get notification emails and how to stop getting them. If you get a lot of site visitors from search engines, the ability to stop the email deluge is important.

How It Works

When the web page is loaded, Search Terms Extractor checks to see if referral information is available. Referral information is the URL of the web page where the browser came from. Not all browsers provide referral information, but most do.

When referral information is available, Search Terms Extractor then checks to see if the referring web site is the search results page of one of the three supported search engines. The software can recognize referrals from Google, Yahoo!, and MSN Live search results.

If yes, any search terms available in the referring URL are extracted. The search terms and the identity of the search engine are stored in PHP variables, which can be used to print them on the web page. The search terms are optionally logged in a file. And they are optionally send in an email to the site owner.

Implementation Instructions for Search Terms Extractor Software

The Search Terms Extractor is written with PHP. It will work only with PHP web pages.

If your web page files all end with .htm, .html, or .shtml, the "Running PHP On Non-.php Web Pages" article may be of interest.

The software can be implemented in 2 or 3 steps:

Upload the processing code as a file on your server. The processing code needs a few easy customizations before uploading.
Insert a line of PHP code into each web page that is to extract the search terms used to find it.
(optional) Insert PHP code into the web page where you want to print the search terms used and/or the identity of the search engine used.

Implementation Step 1: The Processing Code File

This is the file that does the work.

<?php
/***********************************************************

  Search Terms Extractor
  Version 1.0
  March 3, 2008

  Copyright 2008 Bontrager Connection, LLC

  Use of this software is subject to the 
  Bontrager Connection, LLC Software License Agreement.
  http://www.willmaster.com/software/information/license-agreement.php

  When a site visitor's browser arrives from search engine 
  Google, Yahoo!, MSN, or MSN Live, and the browser provides 
  adequate referrer information, the visitor's search terms 
  are determined and made available for use or disposition.

  This is the external file for inclusion into web pages to 
  be monitored. It requires customization. See notes below.

  See the "Search Terms Extractor" article at the 
  Willmaster.com library for implementation instructions: 
  http://www.willmaster.com/library/

  Willmaster.com is the place to get commercial, frugal, and 
  free web site software that is easy to use and makes your 
  business web site work better. http://www.willmaster.com/

***********************************************************/

// Customization Notes --
// There are five customization steps.

// Step 1.
// Between the quotes, specify the email address to send 
//    notification to whenever someone arrives from one 
//    of the search engines. To elect not to receive 
//    notification email, leave it blank (nothing between 
//    the quotes).

$NoticationEmailAddress = "";


// Step 2.
// If you want a log file to be updated whenever someone 
//    arrives from one of the search engines, specify the 
//    directory where the log is to be between the quotes. 
//    The directory may need 777 permissions (depends on 
//    your PHP configuration). If this is left blank, no 
//    log file will be updated.

$LogDirectory = "/SearchTermsLogDirectory";


// Step 3.
// If you want a CSV log file to be updated whenever someone 
//    arrives from one of the search engines, specify the 
//    name the log file is to have between the quotes. If 
//    this is left blank, no log file will be updated.

$LogFileName = "SearchTermsLogFileName.csv";


// Step 4.
// If you want a date stamp to be included with each log 
//    entry, specify "y" or "Yes". 
$DateStamp = "Yes";


// Step 5.
// If you want the search engine name to be included with 
//    each log entry, specify "y" or "Yes". 

$EngineStamp = "Yes";


// Step 6.
// If you want the web page location the visitor arrived at 
//    to be included with each log entry, specify "y" or "Yes". 

$PageStamp = "Yes";

/***********************************************************
    No other customizations are necessary.
***********************************************************/

$tempstring = isset($_SERVER["HTTP_REFERER"]) ? $_SERVER["HTTP_REFERER"] : '';
if( strpos($LogDirectory,'/') === 0 ) { $LogDirectory = $_SERVER["DOCUMENT_ROOT"].$LogDirectory; }
$SearchEngine = $SearchTerm = '';
$DateStamp = strtolower($DateStamp);
$DateStamp = strpos($DateStamp,'y') === 0 ? 'y' : '';
$EngineStamp = strtolower($EngineStamp);
$EngineStamp = strpos($EngineStamp,'y') === 0 ? 'y' : '';

function GetSearchTerm($key,$string)
{
   $terms = '';
   $key .= '=';
   $point = strpos($string,$key);
   if( $point === 0 or $point > 0 )
   {
      $terms = substr( $string, ($point + strlen($key)) );
      $point = strpos($terms,'&');
      if( $point > 0 ) { $terms = substr( $terms, 0, $point ); }
      $terms = urldecode($terms);
   }
   return $terms;
} # function GetSearchTerm()

if($tempstring)
{
   $patternstring = '/https?:\\/\\/[^\\/]*(google|yahoo|msn|live|msnlive)\\./i';
   if( ! preg_match($patternstring,$tempstring) ) { $tempstring = ''; }
}
if($tempstring)
{
   $which = preg_replace($patternstring,'\\1'."\t",$tempstring);
   $atemp = explode("\t",$which);
   $which = $atemp[0];
   switch($which)
   {
      case 'google':
         $SearchEngine = 'Google';
         $SearchTerm = GetSearchTerm('q',$tempstring);
         break;
      case 'msn':
      case 'live':
      case 'msnlive':
         $SearchEngine = 'MSN Live';
         $SearchTerm = GetSearchTerm('q',$tempstring);
         break;
      case 'yahoo':
         $SearchEngine = 'Yahoo!';
         $SearchTerm = GetSearchTerm('p',$tempstring);
         break;
      default: $tempstring = '';
   } # switch($which)
   if( ! $SearchTerm ) { $tempstring = ''; }
}
if($tempstring)
{
   if( preg_match('/@/',$NoticationEmailAddress) )
   {
      $subject = '[STE]-'.$_SERVER['PHP_SELF'];
      $message = "Term: $SearchTerm\nEngine: $SearchEngine\nAt: ".$_SERVER['PHP_SELF'];
      $headers = "From: \"Search Terms Extractor\" <$NoticationEmailAddress>\r\n";
      mail($NoticationEmailAddress,$subject,$message,$headers);
   }
   if( $LogDirectory and $LogFileName )
   {
      $LogDirectory = preg_replace('/\/*$/','',$LogDirectory);
      $fname = "$LogDirectory/$LogFileName";
      $fhandle = fopen($fname,'a');
      if($fhandle)
      {
         $logline = '';
         if($DateStamp)   { $logline .= date('D M d y H:i:s') . ','; }
         if($EngineStamp) { $logline .= "$SearchEngine,";            }
         if($PageStamp)   { $logline .= $_SERVER['PHP_SELF'].',';    }
         if($logline)
         {
            if( preg_match('/"/',$SearchTerm) )
            {
               $SearchTerm = preg_replace('/"/','""',$SearchTerm);
               $SearchTerm = "\"$SearchTerm\"";
            }
         }
         $logline .= $SearchTerm;
         fwrite($fhandle,"$logline\n");
         fclose($fhandle);
      }
      else { echo("<h3>Unable to open file \"$fname\" for writing."); }
   }
   $SearchTerm = preg_replace('/ *site:[^ ]+ */i','',$SearchTerm);
}
?>

You'll notice 6 places to customize:

If you want email notification, provide the email address. Otherwise, leave it blank.
If you want search terms logged, provide the directory where the log file will be.
If you want search terms logged, provide the file name to use.
If logged, specify whether or not to also record a date/time stamp.
If logged, specify whether or not to also record the identify of the search engine.
If logged, specify whether or not to also record the URL of the web page the visitor clicked to.

Specific instructions for each of the above are within the PHP code itself.

Upload the processing code file to your server. For purposes of the examples below, we'll assume the file is named SearchTerms.php and is uploaded into the document root. (The document root is the directory where your web site's main/index file is located.)

Implementation Step 2: The One Line Of Code

Every page that will monitor arrivals from search engine results pages needs to have this line in it somewhere:

Change the processing code file and location if different than above.

The line can be put anywhere on the page. Above the HTML tag, within the HEAD area, even within the BODY area. If search terms or engine identities will be printed on the web page, the PHP line will need to be above their location.

The PHP line can be within other PHP code. However, the leading "<?php" and ending "?>" must be removed from the line.

Implementation Step 3: Printing Terms On The Web Page

This step is optional.

Printing the search word or phrase the site visitor used to find your page, type this in the spot where you want the word or phrase to print:

Similarly, to print the identity of the search engine that was used:

You can put the above right into your regular HTML source code. The search term/engine text can be formatted by surrounding the PHP with HTML or CSS formatting code, like:

It is possible to print certain text with the search term when, and only when, a search term is available. If you are interested in it, click here.

Knowing what people are looking for is so much better than guessing.

Question: Did you find this article interesting and understandable? How can it be improved? Your response is anonymous. When done typing, click anywhere outside the box. [more info]

Please note: Articles on this website are presented "as is". However -

If you have a question about a CGI script, HTML, CSS, PHP, or JavaScript
Ask one of our Experts and you'll have your answer!
Click here for details.

WillMaster > Web Development Tools Reference Library > PHP Code, Tips

Finding something useful?

Subscribe to
the Possibilities Ezine

Most Recent
Software Additions

Go Short URL

Secure View and Remove

Survey Lite

Attention Ticker

Search Terms Extractor

Printing Visitors' Search Terms On the Web Page

Logging Search Terms to a File

Email Notification When Visitor Arrives From Search Engine

How It Works

Implementation Instructions for Search Terms Extractor Software

Implementation Step 1: The Processing Code File

Implementation Step 2: The One Line Of Code

Implementation Step 3: Printing Terms On The Web Page

WillMaster > Web Development Tools Reference Library > PHP Code, Tips

Finding something useful?

Subscribe tothe Possibilities Ezine

Most RecentSoftware Additions

Go Short URL

Secure View and Remove

Survey Lite

Attention Ticker

Search Terms Extractor

Printing Visitors' Search Terms On the Web Page

Logging Search Terms to a File

Email Notification When Visitor Arrives From Search Engine

How It Works

Implementation Instructions for Search Terms Extractor Software

Implementation Step 1: The Processing Code File

Implementation Step 2: The One Line Of Code

Implementation Step 3: Printing Terms On The Web Page

Subscribe to
the Possibilities Ezine

Most Recent
Software Additions