corner1.gif (159 bytes) corner2.gif (159 bytes)

TurboSeek » TurboSeek Documentation >> 6. Running your directory


Turbo Seek Documentation

Index

1. Installation
2. Settings and Options (After installation)
3. The Directory Manager
4. How templates & their variables work
5. Building a Search Index
6. Running your directory
7. Tips, Tricks & Notes
8. Plug-In's

<< Previous Page
Building a search index

Index

Next Page >>
Tips & Tricks

 

6. Running Your Directory

6.0 Overview
6.1 Creating the index/front-page of your directory.
6.2 Running an HTML generated directory.
       6.2.1 Generating the HTML files from the admin control panel.
       6.2.2 Generating the HTML files from the Unix/Linux command prompt.
       6.2.3 If you plan on using SSI on the HTML generated files (advanced users)
       6.2.4 Using cron to generate your HTML files on a scheduled basis
6.3 Running a programmatic directory.
6.4 Using the What's New and Top Hits pages.
6.5 Checking for broken links
       6.5.1 Using Cron to check for broken links
6.6 Data Backups
6.7 Moving your directory to another host or server

 

6.0 Overview

There are two ways you can run your directory. You can run it programmatically (Dynamically) or you can run it from HTML generated files. (Statically) It is highly recommended that you run it from HTML generated files because the load would then be much less on the server that runs your Directory. Your pages will also be Search Engine friendly so that other Search Engines can crawl your Directory/Category pages with ease.

 

6.1 Creating the index/front-page of your directory.

First choose what kind of Directory you will run. You can run two kinds of Directories. You can run a 'Live' Directory or a 'Static' Directory.

  • What is a live directory? (Live)
    The link listings and pages displayed in your Directory is created on the fly via Turbo Seek's cgi scripts. This means that you won't have static HTML files that visitors will load when browsing your Directory but only CGI scripts.

  • What is an HTML generated directory? (Static)
    Turbo Seek will generate HTML files of your entire Directory structure. Note that you have to generate the HTML files first before these links will work. To generate the .html files click on the 'Build Indexes' button and then 'Build Directory Index'. It is recommended that you run a static directory, due to performance reasons. Another reason why we recommend this is because your category/directory pages are static pages where the URLs of these pages are Search Engine friendly. This means that the Crawlers and Spiders (advanced or basic crawlers/spiders) of Search Engines will index your directory/category link pages, as there are no complicated CGI script URLs involved.

To configure Turbo Seek to run a 'Live' or 'Static' Directory, go to the Control Panel >> Settings >> Global Engine Setting >> Look for a setting that reads: 'Would you like to run a static or dynamic Directory?'

Now go to the Control Panel >> Template Manger >> Create/Edit Front Page

Choose a method to create your Front Page.

The next screen will ask you to configure how categories will be displayed on your Front Page.

After configuring how categories will be displayed the next screen will provide you with HTML code or JavaScript code or instructions on what to do.

Below are some notes/instructions for the Front Page creation methods:

  • JavaScript - You create a web page (index.html for example) that contains everything you want your Front Page to contain. (Logo's, graphics, texts, etc). You then get the JavaScript code and Search Box HTML code and place it inside this web page to display your categories and search box. After that you upload the web page you created and use it as your Front Page or Index page of your Directory.  

  • Manual Front Page Creation - Here you just obtain the links to main categories and place them inside a web page (index.html for example). You then upload the web page and use it as your Front Page. Note that category names/links won't automatically update when you make changes to the main categories.  

  • Write Categories to an existing Front Page (index.html) for example - With this method of Front Page creation you create a web page (index.html for example) and place your logo's, texts, graphics, etc. and everything you want to display on this Front Page. Turbo Seek then writes categories (your main categories) to this file.  

  • Server Side Includes - You create a web page (index.html for example) that contains everything you want your Front Page to contain. (Logo's, graphics, texts, etc). You then place SSI include code and the Search Box HTML code inside this web page to display your categories and search box. After that you upload the web page you created and use it as your Front Page or Index page of your Directory.

 

6.2 Running an HTML generated directory.

To configure Turbo Seek to run an HTML Generated Directory (Static) go to the Control Panel >> Settings >> General Configuration >> The first setting you will see will be where you can choose to run a 'Static' or 'Dynamic' Directory.

You can generate the HTML files from the Admin Control Panel or you may generate it from the Unix/Linux command prompt. You can also use the Unix/Linux Cron program to generate the Directory on a scheduled basis.

6.2.1 Generating the HTML files from the Admin Control Panel.

You can generate your entire Directory from the Admin Control Panel by clicking the 'Build Indexes' tab. Then click 'Build Directory Index'.
       
A popup window will appear and your entire directory will be generated as HTML files.

6.2.2 Generating the HTML files from the Unix/Linux command prompt.

You can also generate your Directory from the Unix/Linux command prompt. To do this, telnet/SSH into your server/hosting-account. Go to the directory where admin3.cgi resides in. Type in: perl admin3.cgi

Generating your files from the Unix/Linux command prompt will be much faster than from the browser because there are no browser limitations and connections from the browser to the Admin Control Panel involved.

Command line parameters when generating your directory from the unix command prompt is as follows: (all are optional)

perl admin3.cgi [permission option] [delete]

[permission option] = This parameter can be 755 or left out. When this option is present all files and directories generated will have permissions of 755. If it is left out all files and directories generated will have permissions of 777.

[delete] = This parameter can be DELETE or NO_DELETE - If you specify DELETE the previously generated files and directories will be deleted. If you specify NO_DELETE the previous files and directories that was generated will not be deleted. Note that if you specify NO_DELETE the previously generated files will be overwritten as new files are being generated. If you just execute admin3.cgi and do not provide a DELETE or NO_DELETE parameter you will be prompted to ask if you want to delete previously generated files and directories.

Example 1:
Let's say you want to generate your directories and files from a telnet/ssh session and do not want these files to have permissions of 777. You would type the following:
perl admin3.cgi 755

Example2:
Let's say you run this script from cron and do not want to be prompted when it is executed. (The use of cron is explained a bit later in this section) You would use the following if you want previously generated files to be removed before rebuilding your directory.
perl admin3.cgi DELETE

Example3:
Let's say you run this script from cron and do not want to be prompted when it is executed. (The use of cron is explained a bit later in this section) You would use the following if you want previously generated files not to be removed before building a new directory.
perl admin3.cgi NO_DELETE

Example4:
Let's say you run this script from cron and do not want to be prompted when it is executed. You do not want the files and directories to have permissions of 777.

perl admin3.cgi 755 NO_DELETE
In the above example files and directories generated will have permissions of 755 and the previous files and directories that was generated will not be removed.

perl admin3.cgi 755 DELETE
In the above example files and directories generated will have permissions of 755 and the previous files that was generated will be removed before new files and directories are created.

     
6.2.3 If you plan on using SSI on the HTML generated files (advanced users)

If you plan on using SSI you must not generate the HTML files from the Admin Control Panel. The directory where the built HTML files resides in, (the 'html' directory as explained in the installation) cannot be set to the permission 777 because this would create a security whole.
       
You will have to generate the HTML files from the Unix command prompt, and
the user with whom you are logged in will have to have permission to write to the directory where generated HTML files are written to.
       
In other words, the files that are generated may not be world writable and you will have to generate the HTML files yourself from the command prompt. You may also use Unix/Linux cron for this.

To generate HTML files from the unix command prompt, do the following:
Telnet/SSH into your server/hosting-account. Go to the directory where admin3.cgi resides in. Type in:
perl admin3.cgi 755

The parameter 755 tells the script to create the files with permissions of 755, which will be safe for use with SSI.

Remember to set the file extensions to .shtml in the Admin Control Panel if your host requires the file extensions to be .shtml for using 'server side includes'. You can do this by going to the Admin Control Panel and by clicking on 'Settings & Options'. Then click on 'General Configuration'.

6.2.4 Using cron to generate your HTML files on a scheduled basis

You may use the Unix/Linux cron program to rebuild/generate the HTML files on a daily, weekly or whatever schedule would suite you.
   
You will have to know how the Cron prorgram works. Note that you cannot just execute admin3.cgi with Cron. The reason for this is that admin3.cgi needs the working directory to be the directory in which it resides.

To make the directory in which admin3.cgi resides in, the working directory,  you can create a Unix shell script, (Almost like a dos batch (.bat) file) that executes admin3.cgi
   
The shell script should look something like this:   
   
cd /home/yourdomain/www/cgi-bin/admin # Change to admin3.cgi's working dir
perl admin3.cgi DELETE                         ### Execute admin3.cgi
   
The shell script has to be executable so that cron can execute it.

IMPORTANT
Remember that when you use CRON you will have to supply parameters to admin3.cgi so that you are not prompted to ask whether you want the previously generated files to be deleted. See: '6.2.2 Generating the HTML files from the Unix command prompt' for a complete explanation on which parameters you can use.

 

6.3 Running a programmatic directory.

If you would like to run your Directory programmatically, (Dynamically) you don't have to build a Directory index (Generate HTML files) and all data displayed in your directory would be live.

To run a Programmatic Directory go to the Control Panel >> Settings >> General Configuration

The first option setting you will see is where you can configure Turbo Seek to run a 'Static' or 'Dynamic' Directory.

 

6.4 Using the What's New and Top Hits pages.

The What's New Page lists new link listings added to your Directory/Search-Engine. The Top Hits Page lists the links that received the most clicks in your Directory/Search-Engine.

In order to get/compile a list of the links that is new in your Directory/Search-Engine you have to generate or compile the What's New link listings data. The same goes for the links that received the most clicks.

To compile the What's New Link Listings and Top Hits Link Listings go to the Admin Control Panel and click on Build Indexes  tab - then click on Compile What's New & Top Hits.

You will be prompted to enter a 'from date' to check for new listings added. Add a date. Also complete the rest of the details required and click on the Start Compiling Button.

After data have been compiled you will be able to access the 'What's New' page and 'Top Hits' pages. The URL's of these pages will be provided to you. It might be a good idea to use these URL's throughout the pages of your   Directory/Search-Engine so that users can see what is new and what is the most clicked links in your Directory/Search-Engine.

You can customize the look and feel of the What's New and Top Hits pages by going to the Admin Control Panel. Then click on the 'Template Manager' tab.

You can also generate the what's new and top hit's pages from telnet/ssh which should increase speed from 100% - 1000%
To compile/generate the what's new and top hits data/pages from the command prompt (telnet/ssh), cd to the directory where the admin scripts resides and execute admin12.cgi as follows:
[Some Unix Prompt]> perl admin12.cgi

You can also use cron to generate the Top Hits and What's New pages/data.

admin12.cgi can take command line paramatres for the purposes of using cron.

Below is a description of these parameters:

admin12.cgi [days back] [number of links1] [number of links2]

[days back] = The previous amount of days back you would like to compile new links from.

[number of links1] = How many days back would you like to compile new links from.

[number of links2] = Maximum number of top hits listings to place on 'Top Hits' Page.

Example:

perl admin12.cgi 21 22 23

This will instruct admin12.cgi to compile listings that is not older than 21 days.

The number of links on the 'what's new page' will be 22

The number of links on the 'top hits' page will be 23

General CRON Information

The Unix Cron program is a task scheduler that usually comes with all flavors of unix. If you are on a virtual account you may need special permission from your hosting company to use the Unix Cron program.

It is highly recommended that you know how to use unix cron before attempting to use the unix cron program. You should be able to find help on unix cron all over the web via search engines.

You may use the Unix cron program to compile the what's new and top hits pages on a scheduled basis.
   
Note that you cannot just execute admin12.cgi with cron. The reason for this is that admin12.cgi needs the working directory to be the directory in which it resides.

To make the directory in which admin12.cgi resides in, the working directory, you can create a Unix shell script, (Almost like a dos batch file) which executes admin12.cgi

The creation of the script file should be simple. If you create it in windows and ftp it to your hosting account remember that you will have to make this script file executable by using chmod 755 yourscript

Also remember that you will have to save this script file in unix file format if created and ftp'ed to your server account. Use textpad to save files in unix file format when creating the file in windows.
   
The shell script should look something like this:
   
    ### START OF SHELL SCRIPT
    cd /home/yourdomain/www/cgi-bin/admin  ### Change to admin4.cgi's working dir
    perl admin12.cgi 21 22 23                        ### Execute admin12.cgi   
   ### END OF SHELL SCRIPT     

 

6.5 Checking for broken links.

Do the following to check for broken links:

  • Telnet/SSH into your server/hosting-account

  • Go to the directory where the admin scripts resides in

  • Execute admin19.cgi by typing the following at the unix command prompt: perl admin19.cgi

The link checker will check all url links and generate a report.
   
You can see this and manage the broken links by clicking 'Dead Links' in the directory manager.
   
The link report consists of links that had problems when checking for their existence.

 

6.5.1 Using Cron to check for broken links

You may also use the Unix cron program to check for broken links on a daily, weekly or whatever schedule would suite you.
   
You will have to know how cron works. Note that you cannot just execute admin19.cgi with cron. The reason for this is that admin9.cgi needs the working directory to be the directory in which it resides.

To make the directory in which admin19.cgi resides in, the working directory, you can create a Unix shell script, (Almost like a dos .bat file) that executes admin9.cgi
   
The shell script should look something like this:
   
cd /home/yourdomain/www/cgi-bin/admin ### Change to admin9.cgi's working dir
perl admin19.cgi                      ### Execute admin9.cgi
   
The shell script has to be executable so that cron can execute it.

 

6.6 Data Backups

It is recommended that you backup your data in case of hard drive failures or crashes.
   
To back up all link entries, settings and customizations backup the contents of the 'data' directory. The 'data' directory is the directory that you have been instructed to create when setting up Turbo Seek.

See the installation instructions for an explanation of what the 'data' directory is.

Restoring Backups

When you restore data backups the files will have to have permissions of 777. This is so that the files that you restore are writable by the scripts. Remember that all files and directories will need the permission setting of 777.

Step 1
To restore the backup, first load up the following in your browser:
http://www.yourdomain.com/cgi-bin/tseek/admin/admin.cgi?fct=setperms
The above will ensure that you can overwrite all old files.

Step 2
Now transfer all the data files/directories that you backed up via ftp to your 'data' directory. Replacing all the old files.

Step 3
Set the permissions of all files and directories with it's contents to the permission setting of 777 - It is vital that all directories and files you transferred have the permission setting of 777

 

6.7 Moving your directory to another host or server

Moving your directory-search-engine should be easy and not complicated.
   
Step 1

Setup Turbo Seek like usual on the new machine or hosting account. Once it is running and working, copy/transfer the contents of the 'data' directory on your old host to the 'data' directory of the new host. The 'data' directory is the directory that you have been instructed to create when setting up Turbo Seek.
   
When copying the contents of the 'data' directory from your old host to the new one, you will have to replace the contents of the 'data' directory on the new host. When setting up Turbo Seek on the new host you will have had to create a data directory to get Turbo Seek going.
   
Step 2

Set the permissions of all files and directories in the data directory to 777 - If you don't do this your Directory / Search Engine won't function correctly.
      
Step 3
Load up admin.cgi (the control panel) and you will see that your directory with all links and customizations are in tact on the new host. Your Directory should now be transferred and ready to serve.
   
Remember that you will have to generate new .html files if you chose to run your Directory from generated .html pages. Another way of saying this would be that you will need to rebuild your indexes.

 

 

<< Previous Page
Building a search index

Index

Next Page >>
Tips & Tricks

ffbotl.gif (1447 bytes) FocalMedia.Net Support ffbotr.gif (1456 bytes)