TRICKLE

For a studio album by English trip hop band Olive, see Trickle.

TRICKLE was a file-forwarding service on the BITNET (EARN/NetNorth/GulfNet) network.

When it was created, many sites in Europe only had access to BITNET or compatible networks, and not to the Internet. Therefore, there was a great need to access files that were available only on the Internet. The TRICKLE server allowed anyone on the BITNET-compatible networks to access hundreds of thousands of files from many popular FTP servers around the world.

The TRICKLE server was a 'concentrator' of FTP sites into central servers. Each server had its own (large) cache disk to store files that have been recently ordered by users, and was in communication with all other TRICKLE servers about the contents of its cache. When a user ordered a file that is not in local cache, TRICKLE could usually forward this command to one other TRICKLE server that was known to hold the file. In some cases, "broadcasts" to all other TRICKLEs for this file is necessary, and if the file is new, or accessed very infrequently, it was requested from one of the 3 FTP slave-servers it had. These 3 were special servers who fetch the file via FTP and send it via NJE to TRICKLE.

TRICKLE servers reduced load on FTP sites substantially: Their incoming-to outgoing file rate was usually 1/5. Meaning, each cached file was usually sent out five times during its lifetime in the cache disk.

TRICKLE supported a wide range of delivery methods to the user. Besides the usual email+(UUE, XXE, BTOA, etc.), it could also FTP the files to the user, or use NJE SENDFILE where possible and desired.

It was possible to subscribe to file patterns, such as <MSDOS.VIRUS>SCAN and to have the server deliver you a new copy of the file each time it is made available. A special keyword, '*' could be used instead of the pattern to subscribe to an entire subdirectories, like: <OS2.NEW>*

At its heyday, TRICKLE processed thousands of requests daily, delivering many gigabytes of software to its users.

The server allowed users anywhere around the world to request files from an FTP server on the Internet via a gateway server which was connected to both networks. It was intended to enable the more widespread distribution of (what was then referred to as) Public Domain software from the SIMTEL20 and about a dozen other repositories (hence the command /PDGET, see below).

The software was originally written by Turgut Kalfaoglu at the Turkish national node TREARN (Ege University,Izmir) and was rapidly mirrored at many other BITNET nodes including AWIWUW11 (Austria), BANUFS11 (Belgium), DKTC11 (Denmark), DB0FUB11 or DTUZDV1 (Germany), IMIPOLI (Italy), EB0UB011 (Spain), TAUNIVM (Israel) and others.

To use the service, BITNET users would type a command to their local machine which would forward the remainder of the line as a command to the TRICKLE server. The syntax of the forwarded command was similar to those used on LISTSERV but prefixed with a slash:

TELL TRICKLE AT node /PDGET <directory>file (mode

where node was the BITNET node hosting a TRICKLE gateway; <directory>file was the directory and file required; and (mode specified the file transfer mode (e.g. UUE). On BITNET VAX systems the local syntax was SEND TRICKLE@node.

An example command would look like:

         TELL TRICKLE AT TREARN /PDGET <MSDOS.GAMES>MADMAX.ARC

It also accepted commands sent via email; any number of commands would be put in the body of the message.

The target file was optionally divided into 64Kb chunks after zipping and UUencoding to alleviate congestion on the BITNET network, which used a store-and-forward transmission model where files in transit were sorted into ascending order of size at each intervening node. This enabled very small files to travel reasonably quickly, leaving larger ones to queue for later (e.g. overnight). It was the receiving user's responsibility to piece together the chunks in order for reassembly into the full UUencoded file for decoding and unzipping.

TRICKLE used a distributed cache database - that is, it only kept one copy of each requested file in Europe and each TRICKLE server was aware of each other's files. If anyone asked for any of those files, it would be sent from whichever TRICKLE server happened to have it. This alleviated the load on the saturated international links (esp. Europe-USA), and permitted rapid delivery of popular files. Users could also subscribe to file patterns, using a command like /SUB <MSDOS.VIRUS>SCAN* for example, and each new matching file that appeared in the repository would automatically be sent to the user.

TRICKLE also had the ability to create "newsletters" and mail them out to subscribers - it contained information about each new file that was added to the repository. Likewise, users could send the /NEW command and get a list of the newest additions to the archives.

The server also knew of which servers were closest to their users, and forced users to use their local server. However, if that server was detected to be down, then any other server accepted commands from the users of that region.

TRICKLE was written in REXX programming language (over 40k lines of code) and in Assembler. It ran on IBM VM/CMS systems, Contact: Turgut Kalfaoglu turgut (at) kalfaoglu.com

For this server, Turgut Kalfaoglu was awarded European Academic and Research Network's Software of the Year award.

References: