Monitor FTP | ActiveXperts Network Monitor and Internet Protocols

Introduction to FTP

OVERVIEWING FTP SERVICES

FTP services have been provided with Windows NT since the first version, version 3.1. When IIS 1.0 was released, the FTP service was integrated into IIS, though it continued to run as a separate service. It remained virtually untouched until IIS 4.0 was released. It can still be considered feature-poor, but it now supports multiple virtual servers, remedying a major drawback of earlier versions. Microsoft’s Internet Information Server 4.0 provides a standard set of FTP server features. It has the capability to have a different FTP server directory structure for each IP address on the machine. It also has the capability to use virtual directories and, for the first time in version 4.0, virtual servers. These features allow the FTP file structure not to be bound by the physical file structure on the machine. The FTP service also supports assigning permissions using Windows NT’s built-in NTFS permissions and user accounts.

THE FTP MODEL

The File Transfer Protocol, commonly known as FTP, is another protocol made popular by the public Internet. FTP, as the name implies, allows files of any type to be transferred between a server and a client across a TCP/IP network. If this were the only functionality that FTP provided, it would have been rendered completely obsolete by the HTTP protocol. However, FTP provides for additional functionality that the HTTP protocol does not explicitly accommodate: file and directory management.

FTP, like HTTP and many other Internet protocols, makes use of English text commands between the client and server. This makes it an easy protocol to troubleshoot and to write programs to utilize. Another advantage of FTP is that it acts as a layer of abstraction between the client and the server’s operating system. While different vendor’s implementations of FTP may vary to some degree, an FTP server looks like any other FTP server, whether it is based on Windows NT or a flavor of UNIX.

FTP has existed in one form or another for more than thirty-five years. The current implementation of FTP is defined by RFC 959, though more than fifty RFCs relate to FTP in one way or another.

FTP Servers

Almost all network operating systems support FTP as a server. Windows NT, the focus of this book, has supported FTP services since version 3.1 - long before an HTTP server was available. When Windows NT 3.1 was created, the FTP server was an attempt at UNIX compatibility and interoperability. Today, the FTP service is integrated into Microsoft’s Internet Information Server, which also includes HTTP services. Indeed, Microsoft has slowed development of its FTP service in favor of more modern technologies–the version of FTP included with IIS 2.0 and IIS 3.0 was almost identical to the original FTP released with NT 3.1.
As with most things on the Internet, Windows NT-based systems make up the minority. The majority of FTP sites on the Internet are hosted on UNIX-based systems. Nonetheless, the beauty of FTP is that any client can use any server - making the operating system unimportant.

FTP Clients

Every major desktop operating system includes support for the FTP protocol. Windows NT includes two separate programs for transferring files with FTP:

• FTP: A command-line utility that allows full FTP functionality. It is not as commonly used as other alternatives because it has a clumsy user interface. However, it is flexible and can be used as part of batch files.
• Internet Explorer: A graphical utility that supports only a subset of FTP’s functionality. In normal use, it only supports downloading files with the anonymous user account. It allows downloading a single file when provided with a full URL in the form of ftp://server.net/file.zip, and it allows navigation of the server’s file system when provided with a URL that does not include a filename.

Other common FTP clients include CuteFTP, WSFTP, and Netscape Navigator.

Working with FTP

If your work involves the Internet, you have to work with the FTP protocol. For network administrators, you have to understand the protocol well enough to configure your routers, proxy servers, and/or firewalls to handle it correctly. For ISPs, you have to know the implications of the protocol to troubleshoot problems that your clientele may have. For Webmasters, you have to understand the protocol to troubleshoot problems and to determine when it is appropriate to use FTP, and when it is not.

Troubleshooting

Troubleshooting FTP is a pleasure, at least compared to troubleshooting some other protocols. FTP is a very understandable protocol because all control communications occur in clear text. Problems you will run into using the FTP protocol usually fall into authentication or permissions categories.

Authentication problems include users who cannot log on, users who can log on but shouldn’t, and users who are authenticated but not authenticated with the correct permissions. One of the most common authentication problems with any protocol is the mistyped password. These problems carry over into FTP, but fortunately they are easy to troubleshoot. Consider the following scenario: You have just handed off the username and password to a new user, who is having problems FTPing into the server. Using a protocol analyzer, you could watch the commands as they are submitted to the FTP server, and verify that the user is typing the correct username and password. Later in this section I will detail exactly what you need to look for in the packets to distinguish the user’s username and password from all the other information being carried on the network.

Along the same lines, malformed usernames are a common source of authentication problems. It may be as simple as the user mistyping his or her name, or it may be more complicated. For example, some proxy servers simply strip out the backslash character from an FTP username, making it impossible for the user to log into FTP using a Windows NT domain user account (which must be submitted in the form DOMAIN\USERNAME). No matter what the problem, determining the exact data being submitted to the FTP server can, at the very least, eliminate a few possible sources of problems. This same technique can be used to verify that a user is logging onto FTP with the correct username and password and not using an FTP client configured to automatically log on with the anonymous user.

Permission problems are another common complaint of FTP users. These can be difficult to diagnose, because FTP client applications may replace the error message returned by the FTP server with something more user-friendly, such as Error: Command Failed. With the use of a protocol analyzer, you can determine exactly what the client is attempting to do with the server and exactly what error messages the server is returning to the client. By analyzing the protocol communications themselves, you can reduce the amount of time required to decode messages.

Writing scripts and applications that use FTP

Many administrators may wish to automate certain tasks that use the FTP protocol. For example, an administrator may wish to write a script that will download several files from an FTP server on a regular basis. The administrator may even wish to create a script that checks for updated files in a directory and downloads them as needed.

Fortunately, there are many simple ways to script FTP commands. The simplest method is to use the FTP command with the -s operator. By executing a command in the form FTP -s:file.txt, the administrator can automate simple FTP tasks such as uploading or downloading a previously determined set of files.

If you require greater flexibility, Perl has a full library to simplify FTP communications. Most programming languages that include network support will include libraries for the FTP protocol, largely because of the long history and great stability of the protocol. If you are a truly gung-ho programmer, try writing FTP applications that work at the sockets level. Sure, you will waste a lot of time that could be used productively, but you will develop an extremely deep understanding of the protocols involved!

FTP SERVICES

This section covers the actual commands and responses exchanged between FTP clients and servers. The information contained in this section is very detailed and critical to anyone performing network analysis on the FTP protocol, anyone who writes FTP clients or servers, or anyone who scripts FTP communications.

Terminology

The term DTP refers to the data transfer process, the method used to control the data transfer connection.

The term PI refers to the protocol interpreter. You will see the terms "server-PI" and "user-PI." The server protocol interpreter is the process listening on TCP port 21 on the FTP server. It must be able to respond to standard FTP commands such as USER and PASS. The server-PI has control over the server-DTP. The server-PI uses the server-DTP to transfer data to the user. The user-PI is typically a client-side application that knows how to speak the FTP protocol. The user-PI controls the user-DTP.

Common Commands

The commands listed in this section are probably all the FTP commands you need to know. Each one of these commands is commonly used by FTP clients and should be supported in some fashion by all FTP servers.

USER (USER NAME)

All FTP communications begin with the USER command. This command takes a single argument: the username the client wishes to be authenticated with. In a Windows NT environment, this may include both the domain name and the username. For example, when logging onto an NT server that is a member of a domain, the client may transmit the command:

USER domain\username

The most common argument to the USER command is anonymous. Anonymous logons to FTP are common on the Internet, where a large percentage of FTP servers carry information for the general public.

PASS (PASSWORD)

Generally the second command transmitted by a client to the server, the PASS command carries as an argument the password for the user already specified by the USER command. This command really is as simple as it seems: There is no encoding or encryption of the password, it is simply clear text. An example of transmitting a password from a client to a server:

PASS elvislives

CWD (CHANGE WORKING DIRECTORY)

The CWD command changes the directory the FTP server is working with. The sole argument for this command is the new directory, in either absolute form or relative form. Examples of both of these forms are given here:

CWD /usr/root
CWD /usr/root/
CWD documents

The first command changes the current directory to /usr/root, regardless of what the current directory is. The second command illustrates an optional slash at the end of the directory name. The third command moves into a child directory of the current directory named documents; it only works from directories that have a subdirectory named documents.

QUIT (LOG OUT)

The QUIT command is sent to the server to indicate that the FTP session is over. This command takes no arguments.

RETR (DOWNLOAD)

When the RETR command is issued from the client, a data transfer connection is established. The RETR command takes as an argument the path to the file to be transferred. For example, to use the RETR command to transfer the file /documents/file.html, issue the command:

RETR /documents/file.html

STOR (UPLOAD)

Similar in function and execution to the RETR command, the STOR command sends a file from the client to the server. The only argument for the STOR command is the destination location on the server. If the file already exists in the destination directory, it is automatically overwritten. To upload the file file.html to the /documents directory, issue the command:

STOR /documents/file.html

REST (RESTART)

The REST command is used to continue a session that has been interrupted. The REST command has an argument, an integer, that represents the position in the file where transfer should begin. For example, to restart a transfer at byte 4096 in a file, the client would issue the following commands to the server:

REST 4096
RETR /documents/file.zip

It is important to understand that the command that follows the REST command must be a transfer of some kind, either a STOR or a RECV.

RNFR, RNTO (RENAME FROM, RENAME TO)

The RNFR and RNTO commands always work together. Rather than issuing two arguments to a single command, as with the MS-DOS command rename file1.txt file2.txt, the rename function is broken into two individual commands. For example:

Client> RNFR file1.txt
Server> 350 File exists, ready for destination name
Client> RNTO file2.txt
Server> 250 RNTO command successful.

These two commands must always be used together.

ABOR (ABORT)

The ABOR command cancels any command that is currently being executed on the server. If data is currently being transferred in a data session, it is cancelled. This command takes no arguments.

DELE (DELETE)

The DELE command removes a file on the server, assuming the user has the appropriate permissions. It takes a single argument: the relative or absolute path to the file to be removed. For example, the following command will remove the file /documents/file.htm from the server:

DELE /documents/file.htm

RMD (REMOVE DIRECTORY)

The RMD command is similar to the DELE command, except that it removes an empty directory instead of a file. To remove the directory documents, first make sure that it contains no files whatsoever. Then issue the command:

RMD /documents

Please note that the RMD command does not let you remove virtual directories created on an IIS server, only physical directories.

MKD (MAKE DIRECTORY)

To create a directory on an FTP server, issue the MKD command with either a relative or absolute path given as the only argument. The path given must include only a single directory that does not yet exist. For example, you cannot issue the MKD command to create two nested directories at once. To create the directory production within the already existing documents directory, you would issue the command:

MKD /documents/production
PWD (PRINT WORKING DIRECTORY)

The client uses the PWD command to determine the current active directory on the server. The active directory is the default location for all commands that do not include an absolute directory path. In theory, this would only be necessary when a client first establishes a connection to the server; after that, file system navigation is up to the client, so the client should be able to compute its working directory from the commands it issues to the server. This command takes no arguments and forces the server to return a 257 message with the directory path listed as an argument.

LIST (DIRECTORY LISTING)

The LIST command causes a data transfer to occur that will contain the directory listing for the current working directory. An absolute or relative path can be given as an optional argument if a directory listing for another directory is desired.

SYST (EXECUTE SYSTEM COMMAND)

The SYST command is used to determine what operating system the remote FTP server is running. It does not tell you what specific FTP service it is running, which is more useful information. The SYST command takes no arguments.

NOOP (NO OPERATION)

This command does nothing except stall the remote server. In many cases, the remote FTP server will close the connection after a certain amount of time has passed with no activity from the client. The NOOP command can be issued to reset this time-out value. Please note that not every Web server will respond the same way: Many will simply ignore the command and will not reset the time-out value.

CDUP

The CDUP command changes the current working directory to its parent directory. It is equivalent to the CD .. command in MS-DOS. Indeed, entering CD .. into the command-line FTP client included with Windows NT will cause it to issue a CDUP command to the server.

This command is considered optional in the FTP standards - any client could perform the equivalent of the CDUP command by issuing a CWD command to the absolute path of the parent directory of the current working directory or by issuing the CWD .. command.

PASV

The PASV command is used when the client wishes to connect to the server to transfer data. This is the method the majority of FTP clients use to transfer data. The server normally returns a 227 message, which includes information about the IP address of the FTP server and the port number to connect to. This command takes no arguments.

PORT

The PORT command is used to set the data communication port and IP address. The argument is somewhat counterintuitive: It is composed of both the IP address and the TCP port number, broken into eight-bit fields and separated by commas. The IP address is obvious when examining this command, but most people will need to pull out a calculator to compute the TCP port number. For example, the command:

PORT 10,10,3,17,14,255

means that the IP address 10.10.3.17 should be used with the TCP port number 3839. To calculate the TCP port number, use the formula b5 * 256 + b6, where b5 and b6 are the fifth and sixth bytes respectively. In this case, b5 would be 14 and b6 would be 255.

TYPE

Most FTP clients and servers support transferring files either as ASCII text or as binary. To change types, send one of these two commands:

TYPE A
TYPE I

The argument A means an ASCII file will be transmitted. The argument I means a file image will be transmitted. It is always safe to send a file as an image. Two other types of transfers, E and L, are not commonly used.