Chip,
Are you absolutely certain the performance bottleneck is specifically in the
FTP libraries?
You may want to separate out the routines logically and do some benchmarking to
further isolate the behavior. So you can actually see where the time is being
spent. Just putting in log4j and configuring it to log time can be sufficient
for this.
But before you do that: my concern about the code below is the usage of
FileInputStream alone - each read called on this class generates an operating
system call, reading a byte at a time. You may want to try something more like:
FileInputStream fis = new FileInputStream(someFile);
BufferedInputStream bis = new BufferedInputStream(fis);
which will generate much less native OS access.
You may also want to turn on verbose garbage collection and see if you are
getting pauses because of the collector. In that case you would either want to
bump up the amount of available memory via -Xms, or depending on the
environment you are running in (client vs server, OS, and hardware) try tuning
the collector based on the lifetimes of the objects being created.
May or may not work out, but I always like to work through those things as
methodically as possible.
HTH,
Greg Nieman
p.s.
Say Hi to Cary and Kevin for me.
>
> From: "Chip Dunning" <cdunning@xxxxxxxxxxxx>
> Date: 2004/07/09 Fri PM 03:58:36 GMT
> To: "CinJUG (E-mail)" <users@xxxxxxxxxx>
> Subject: RE: [cinjug-users] FTP
>
> I tried both the core libraries and the ones in the Apache Commons project. I
> got the same results from both sets. Currently I am using the Apache Commons
> version because it was more stable for connections over time. The JVM is
> 1.4.2_03 and the OS is Windows2000 on both ends.
> All of the PCs involved use MS basic FTP capability except for one
> client which uses Serv-U so that we can limit its available bandwidth as
> other more important traffic also flows down the same pipe. We checked that
> one slow pipe wasn't messing up the work for all involved by only FTPing
> along the 100Mb local LAN (and its still 3x slower than a simple Command FTP).
>
> I have include a partial version of the class below that does the actual FTP
> work. We FTP into a temp directory, then execute a rename command to move the
> file into the correct directory. This is because we have a client on the
> other side scanning this directory for files - which it will place into the
> correct local location.
>
>
> Chip
> -----
> "The reason the mainstream is thought of as a stream is because it's so
> shallow" --George Carlin
>
>
> ===============[ code snippet ]=======================
>
> import org.apache.commons.net.ftp.FTP;
> import org.apache.commons.net.ftp.FTPClient;
> import org.apache.commons.net.ftp.FTPConnectionClosedException;
> import org.apache.commons.net.ftp.FTPReply;
>
>
> /**
> * @author cdunning
> * @version 0.0
> *
> * This product includes software developed by the
> * Apache Software Foundation (http://www.apache.org/).
> */
> public class FTPMover extends Thread
> implements Observer {
>
> [snipped a bunch of setup/database stuff]
>
>
> /** Transfer the file
> * @param srcFile File to ftp
> * @return true is the ftp is successful
> */
> private boolean ftpFile(File srcFile) {
> theLogger.finest("Attempting to ftp " + srcFile.getName());
> boolean status = false;
>
> FTPClient ftp = new FTPClient();
>
> DataModel dm = Toolbox.getInstance().getDataModel();
> String rcvdPath = dm.getReceiveDirectory();
> String tempPath = dm.getTempDirectory();
>
> try {
> ftp.setDefaultTimeout(dm.getFtpTimeout());
> ftp.enterLocalActiveMode();
>
> ftp.connect(ipAddress, 21);
> customerData.setReplayStatus("Connected");
>
> if (!ftp.login(dm.getFTPUser(), dm.getFTPPassword())) {
> ftp.logout();
> customerData.setReplayStatus("Login Failed");
> theLogger.log(Level.SEVERE, ipAddress + " failed login");
> throw new IOException();
> }
> customerData.setReplayStatus("Logged In");
>
> ftp.setFileTransferMode(FTP.STREAM_TRANSFER_MODE);
> ftp.setFileType(FTP.BINARY_FILE_TYPE);
> ftp.setSoTimeout(600000);
>
> if (!FTPReply.isPositiveCompletion(ftp.getReplyCode())) {
> ftp.logout();
> System.err.println("FTP server refused connection.");
> customerData.setReplayStatus("Disconnected");
> theLogger.log(Level.SEVERE, ipAddress
> + " refused ftp connection");
> throw new IOException();
> }
>
> customerData.setReplayStatus("FTP " + srcFile.getName());
> InputStream input = new FileInputStream(srcFile);
> ftp.storeFile(tempPath + File.separator + srcFile.getName(),
> input);
> input.close();
> customerData.setReplayStatus("FTP Successful");
>
> sleep(1500);
> ftp.rename(tempPath + File.separator + srcFile.getName(),
> rcvdPath + File.separator + srcFile.getName());
> ftp.logout();
> customerData.setReplayStatus("Logged Out");
> status = true;
>
> } catch (InterruptedIOException e) {
> theLogger.log(Level.SEVERE, ipAddress + " FTP Timeout ("
> + e.bytesTransferred + " bytes transferred)", e);
> errorLog.append("FTP Timeout (" + srcFile.getName() + ")\n");
> status = false;
>
> } catch (FTPConnectionClosedException ex) {
> theLogger.log(Level.SEVERE, ipAddress + " severed connection on "
> + srcFile, ex);
> errorLog.append("Server FTP closed connection ("
> + srcFile.getName() + ")\n");
> status = false;
>
> } catch (IOException ex) {
> customerData.setReplayStatus("Error");
> theLogger.log(Level.SEVERE, ipAddress + " connection failure on "
> + srcFile.getName(), ex);
> errorLog.append("FTP Connection Error (" + srcFile.getName()
> + ")\n");
> status = false;
>
> } catch (InterruptedException e) {
> theLogger.log(Level.FINER, ipAddress + " sleep disrupted");
> status = false;
>
> } finally {
> if (ftp.isConnected()) {
> try {
> ftp.disconnect();
> } catch (IOException f) {
> theLogger.warning(ipAddress + " ftp failed to disconnect");
> }
> }
> }
>
> return status;
> }
>
>
> /**
> * Depending on what happened during the last time this program
> * was run it may have left files in the temp directory. This
> * will clean up any files left behind.
> *
> * @return true if all the temporary directory is clean after this runs
> */
> private boolean cleanupTempDirectory() {
> FTPClient ftp = new FTPClient();
>
> try {
> DataModel dm = Toolbox.getInstance().getDataModel();
>
> ftp.setDefaultTimeout(dm.getFtpTimeout());
> ftp.enterLocalActiveMode();
>
> ftp.connect(ipAddress, 21);
> customerData.setReplayStatus("Connected");
>
> if (!ftp.login(dm.getFTPUser(), dm.getFTPPassword())) {
> ftp.logout();
> customerData.setReplayStatus("Login Failed");
> theLogger.log(Level.SEVERE, ipAddress + " failed login");
> throw new IOException();
> }
> customerData.setReplayStatus("Logged In");
>
> ftp.setSoTimeout(60000);
>
> if (!FTPReply.isPositiveCompletion(ftp.getReplyCode())) {
> ftp.logout();
> System.err.println("FTP server refused connection.");
> customerData.setReplayStatus("Disconnected");
> theLogger.log(Level.SEVERE, ipAddress
> + " refused ftp connection");
> throw new IOException();
> }
>
> ftp.changeWorkingDirectory(dm.getTempDirectory());
> String[] files = ftp.listNames();
> customerData.setReplayStatus("Cleaning temp");
>
> for (int i = 0; i < files.length; i++) {
> ftp.deleteFile(files[i]);
> theLogger.log(Level.INFO, ipAddress + " Deleting " + files[i]
> + " from temp directory\n");
> }
>
> } catch (InterruptedIOException e) {
> theLogger.log(Level.SEVERE, "FTP Timeout", e);
>
> } catch (IOException ex) {
> errorLog.append("FTP IO error in temp directory\n");
> theLogger.log(Level.SEVERE, ipAddress + " IOError", ex);
> return false;
>
> } finally {
> customerData.setReplayStatus("Logging out");
> if (ftp.isConnected()) {
> try {
> ftp.disconnect();
> } catch (IOException f) {
> theLogger.warning(ipAddress + " ftp failed to disconnect");
> }
> }
> }
>
> customerData.setReplayStatus("Finished");
> return true;
> }
>
> }
>
> ---------
> You may unsubscribe from this mailing list
> by sending a blank email addressed to:
> users-unsubscribe@xxxxxxxxxx
>
> --
> Find additional help by sending a blank email
> addressed to:
> users-help@xxxxxxxxxx
>
>
|