(Get Attention Ticker for your web site)

Backup/Restore & Export/Import & Server Move Perl DBM Database Tools

By Will Bontrager

Imagine opening a database file and manipulating it as easily as you would a normal Perl variable. In fact, you can use the same code as you would if it was not a database file.

It's called tieing a variable to a file. A choice of several Perl modules make this easy to do.

The file the modules create is a DBM database (DBM is an acronym for DataBase Manager).

I use the feature a lot. Files many megabytes in size can be manipulated just as if they were actually residing in the script's memory variables. And you don't have to pay any attention to file updates; once the variable is tied to the file, the module being used does all file updates automatically.

But there is a drawback. Because DBM files are not plain text files, backups and server moves can be a hassle.

When backing up or restoring the DBM files themselves, FTP downloads and uploads must be done as if they were binary files. In the case of a server move, the new server's configuration might be different than the one on the old server, making the moved file useless.

This blog post will provide the tools to back up and restore Perl DBM files and the means to move the information in the files to other servers.

Backing up is exporting the data to a plain text file. Restoring is importing the data into the DBM file. FTP file downloads and uploads of these exported files must be done as ASCII/plain text files.

For server moves, simply export the data, move the plain text file to the new server, then import the data. A similar procedure can copy the data from one DBM file type to another DBM file type on the same server — when a third-party script needs access to the DBM database, for example.

[click here to reveal more geeky things about DBM files]

If your script uses dbmopen() instead of tie(), and you're unsure which DBM module is utilized, use Master Pre-Installation Tester or other means to determine which DBM modules are on your server. Possibilities are:

SDBM_File
DB_File
GDBM_File
NDBM_File
ODBM_File

(Module SDBM_File comes with perl.)

Once you have a list, make a backup copy of the original DBM files. Then use the backup script (below in the blog post) with each module until you get intelligible data in the exported file. Now you know which DBM module your script uses.

(If your script uses dbmopen() or the AnyDBM_File module, the effective DBM module could change if the hosting company reconfigures Perl or adds/removes other DBM modules.)

When exporting for backups or moves, the data is stored in a flat file database, one line per record. Thus, the return and linefeed characters need to be replaced with something else. The backup and restore scripts (below) allow you to specify what characters to use as replacements.

This script can back up/export the information in a DBM file to a plain text flat file database:

#!/usr/bin/perl
# Copyright 2006 Bontrager Connection, LLC
# Must agree with the software license agreement at 
#   http://www.willmaster.com/master/LicenseAgreement.html
use strict;
use Fcntl;


# The file name of the plain text file (include path if applicable).

my $PlainTextFile = 'directory/file.txt';


# The delimiter to use in the plain text file to separate the 
#   DBM key from the value. It needs to be a character never 
#   used in the DBM key. It may not be ASCII character 10 or 13 
#   (return/line feed characters). Any character 1 through 9 is 
#   probably okay.

my $KeyValueDelimiter = chr(1);

# The character to replace ASCII character 10 (return) with. Any 
#   character 1 through 9 is probably okay, so long as it is not 
#   identical to the character assigned to the $KeyValueDelimiter 
#   variable.

my $ReturnCharacterReplacement = chr(2);


# The character to replace ASCII character 13 (line feed) with. 
#   Any character 1 through 9 is probably okay, so long as it 
#   is not identical to the character assigned to the 
#   $KeyValueDelimiter or $ReturnCharacterReplacement variables.

my $LinefeedCharacterReplacement = chr(3);



#### All lines above here should identical in  ####
####  both backup.cgi and restore.cgi scripts. ####
###################################################
###                    #####                    ###
##                      ###                      ##
#                        #                        #
##                      ###                      ##
###                    #####                    ###
###################################################
#### The following might be different between  ####
####  the backup.cgi and restore.cgi scripts.  ####


# The variable name to tie to when the DBM file is opened.
#   The variable name may be changed if all corresponding 
#   references in the rest of the script are also changed.

my %HashForTie;



# The use ... DBM module line goes here, if your 
#   working script uses it. (If dbmopen() is used 
#   to tie the variable, use ..variable name isn't 
#   always necessary, but put it here if the working 
#   script uses it.)
# Your working script might use one (almost always no 
#   more than one) of tne of the following use ... DBM 
#   modules:
#     use AnyDBM_File;
#     use SDBM_File;
#     use DB_File;
#     use GDBM_File;
#     use NDBM_File;
#     use ODBM_File;
#   AnyDBM_File isn't actually a DBM database module, but 
#     selects a DBM module to use. 

### If applicable, put the use ... DBM module line below. ###

use NDBM_File;




# Here, open the DBM file (with tie() or dbmopen()), 
#   using the same syntax as your working script. It 
#   is assumed %HashForTie is the tied variable; if not, 
#   all references to "HashForTie" in this script must 
#   be changed accordingly. ("HashForTie" is preceded 
#   with a "%" or "$" character.)
# Here is an example of a tie() line:
#     tie(%HashForTie,'NDBM_File',"directory/dbm",O_RDWR|O_CREAT,0666);

### Put the tie() or dbmopen() line below. ###

tie(%HashForTie,'NDBM_File',"directory/dbm",O_RDWR|O_CREAT,0666);

# Note: When specifying the file name of the DBM file, omit 
#       any file name extension. Let Perl and the modules 
#       determine the extension, if any. (Not all DBM files 
#       have file name extensions.)



# This does the work:

open W,">$PlainTextFile";
while(my($k,$v) = each %HashForTie)
{
	$k =~ s/\r/$ReturnCharacterReplacement/sg;
	$k =~ s/\n/$LinefeedCharacterReplacement/sg;
	$v =~ s/\r/$ReturnCharacterReplacement/sg;
	$v =~ s/\n/$LinefeedCharacterReplacement/sg;
	print W "$k$KeyValueDelimiter$v\n";
}

print "Content-type: text/html\n\nDONE";
# end of script

To make a backup file, customize, install, and run the above backup.cgi script.

Download the resulting plain text file for safekeeping. If moving the database to a new server, upload the plain text file to the new server.

This script can restore/import the information in a flat file database created with the restore.cgi script into a DBM database:

#!/usr/bin/perl
# Copyright 2006 Bontrager Connection, LLC
# Must agree with the software license agreement at 
#   http://www.willmaster.com/master/LicenseAgreement.html
use strict;
use Fcntl;


# The file name of the plain text file (include path if applicable).

my $PlainTextFile = 'directory/file.txt';


# The delimiter to use in the plain text file to separate the 
#   DBM key from the value. It needs to be a character never 
#   used in the DBM key. It may not be ASCII character 10 or 13 
#   (return/line feed characters). Any character 1 through 9 is 
#   probably okay.

my $KeyValueDelimiter = chr(1);

# The character to replace ASCII character 10 (return) with. Any 
#   character 1 through 9 is probably okay, so long as it is not 
#   identical to the character assigned to the $KeyValueDelimiter 
#   variable.

my $ReturnCharacterReplacement = chr(2);


# The character to replace ASCII character 13 (line feed) with. 
#   Any character 1 through 9 is probably okay, so long as it 
#   is not identical to the character assigned to the 
#   $KeyValueDelimiter or $ReturnCharacterReplacement variables.

my $LinefeedCharacterReplacement = chr(3);



#### All lines above here should identical in  ####
####  both backup.cgi and restore.cgi scripts. ####
###################################################
###                    #####                    ###
##                      ###                      ##
#                        #                        #
##                      ###                      ##
###                    #####                    ###
###################################################
#### The following might be different between  ####
####  the backup.cgi and restore.cgi scripts.  ####


# The variable name to tie to when the DBM file is opened.
#   The variable name may be changed if all corresponding 
#   references in the rest of the script are also changed.

my %HashForTie;



# The use ... DBM module line goes here, if your 
#   working script uses it. (If dbmopen() is used 
#   to tie the variable, use ..variable name isn't 
#   always necessary, but put it here if the working 
#   script uses it.)
# Your working script might use one (almost always no 
#   more than one) of tne of the following use ... DBM 
#   modules:
#     use AnyDBM_File;
#     use SDBM_File;
#     use DB_File;
#     use GDBM_File;
#     use NDBM_File;
#     use ODBM_File;
#   AnyDBM_File isn't actually a DBM database module, but 
#     selects a DBM module to use. 

### If applicable, put the use ... DBM module line below. ###

use NDBM_File;



# Here, open the DBM file (with tie() or dbmopen()), 
#   using the same syntax as your working script. It 
#   is assumed %HashForTie is the tied variable; if not, 
#   all references to "HashForTie" in this script must 
#   be changed accordingly. ("HashForTie" is preceded 
#   with a "%" or "$" character.)
# Here is an example of a tie() line:
#     tie(%HashForTie,'NDBM_File',"directory/dbm",O_RDWR|O_CREAT,0666);

### Put the tie() or dbmopen() line below. ###

tie(%HashForTie,'NDBM_File',"directory/dbm",O_RDWR|O_CREAT,0666);

# Note: When specifying the file name of the DBM file, omit 
#       any file name extension. Let Perl and the modules 
#       determine the extension, if any. (Not all DBM files 
#       have file name extensions.)



# This does the work:

open R,"<$PlainTextFile";
while(<R>)
{
	chomp;
	my($k,$v) = split /\Q$KeyValueDelimiter\E/,$_,2;
	$k =~ s/\Q$ReturnCharacterReplacement\E/\r/sg;
	$k =~ s/\Q$LinefeedCharacterReplacement\E/\n/sg;
	$v =~ s/\Q$ReturnCharacterReplacement\E/\r/sg;
	$v =~ s/\Q$LinefeedCharacterReplacement\E/\n/sg;
	$HashForTie{$k} = $v;
}

print "Content-type: text/html\n\nDONE";
# end of script

To restore the backup file, customize, install, and run the above restore.cgi script.

July 9, 2006

Please note: Articles on this website are presented "as is". However -

If you have a question about a CGI script, HTML, CSS, PHP, or JavaScript
Ask one of our Experts and you'll have your answer!
Click here for details.

Rate this blog post.
No page reload!

WillMaster > Blog > CGI

Subscribe to
the Possibilities Ezine

RSS feed links

Blog Article Categories

Backup/Restore & Export/Import & Server Move Perl DBM Database Tools

WillMaster > Blog > CGI

Subscribe tothe Possibilities Ezine

RSS feed links

Blog Article Categories

Backup/Restore & Export/Import & Server Move Perl DBM Database Tools

Subscribe to
the Possibilities Ezine