Effective Block for Browsers and Bots
There is a way to block all browsers and bots from specific directories – without resorting to password protection (which might be guessed). It's an effective block.
Yet, PHP and Perl CGI software installed on the same domain can still access the files.
The protection consists of 1 line of code, 3 words. The 1 line of code is further below, in "The Code" section of the article.
Requirement
It's a nice little trick done with the .htaccess file. Which means your site needs to be .htaccess-capable – generally, Unix/Linux hosting.
The .htaccess file's line endings need to be compatible with Unix/Linux. If you get an Internal Server Error, incorrect line endings may be the reason.
If you don't have software to put specific line endings into files, upload the .htaccess file with FTP and verify the file is transferred as a plain text file instead of binary. Transferring as plain text causes the FTP software to change the line endings of the file to the type expected by the destination.
It may help to name the file htaccess.txt (which generally transfers as a plain text file) and, once the file is on the server, rename it to .htaccess
Deprecation
(This section has some technical information. It's purpose is disclosure. If you want to skip the section, or if it seems hard to grasp, realize that it's not something you have to be immediately concerned about.)
The Apache server software module that processes the 1-line of code you'll be using is being deprecated. It's still good, even in the latest Apache version.
However, the Apache organization is likely to release a version sometime in the future that no longer supports the current module, not even for backward compatibility. We don't know when that will be. Once they do, then sooner or later your hosting company will upgrade the Apache software on your server to that version.
When that happens, it is expected browsers and bots that try to access the protected directory, with the no longer recognizable code in the .htaccess file, will encounter an "Internal Server Error" instead of the current "Forbidden" status message.
At that point, you may decide to replace the 1 line of code presented in this article with code using a new module's language.
The Code
Here's the line of code to block browser and bot access to a directory. Put it into the .htaccess file of the directory to be blocked.
deny from all
That's it. All the browsers and bots are blocked!
Server Software
PHP and Perl CGI software installed on the same domain can still access the files in the protected directory.
When server software needs to work with a confidential file in the document area, this method of protecting the file works a treat.
Will Bontrager
