Ftp In Aws Lambda - Issues Downloading Files (async/await)

I have been struggling with various FTP Node modules to try and get anything working in AWS Lambda. The best and most popular seems to be 'Basic-FTP' that also supports async/await

Solution 1:

The problem is that you are getting lost when creating an async function inside your handler. Since example() is async, it returns a Promise. But you don't await on it, so the way it has been coded, it's kind of a fire and forget thing. Also, your Lambda is being terminated before your callbacks are triggered, so even if it got to download you would not be able to see it.

I suggest you wrap your callbacks in Promises so you can easily await on them from your handler function.

I have managed to make it work: I have used https://dlptest.com/ftp-test/ for testing, so change it accordingly. Furthermore, see that I have uploaded the file myself. So if you want to replicate this example, just create a readme.txt on the root of your project and upload it. If you already have this readme.txt file on your FTP server, just delete the line where it uploads the file.

Here's a working example:

const ftp = require("basic-ftp");
const fs = require("fs");
constFileNameWithExtension = "readme.txt";

module.exports.hello = async (event) => {

  const client = new ftp.Client();
  try {
    await client.access({
      host: 'ftp.dlptest.com',
      user: 'dlpuser@dlptest.com',
      password: 'puTeT3Yei1IJ4UYT7q0r'
    });
    console.log(await client.list());
    await client.upload(fs.createReadStream(FileNameWithExtension), FileNameWithExtension)
    await client.download(fs.createWriteStream('/tmp/' + FileNameWithExtension), FileNameWithExtension);
  }
  catch (err) {
    console.log('logging err')
    console.log(err);
  }
  client.close();

  console.log(awaitreaddir('/tmp/'))

  console.log(awaitreadfile('/tmp/', FileNameWithExtension))

  return {
    statusCode: 200,
    body: JSON.stringify({message: 'File downloaded successfully'})
  }

};

constreaddir = dir => {
  returnnewPromise((res, rej) => {
    fs.readdir(dir, function (err, data) {
      if (err) {
        returnrej(err);
      }
      returnres(data)
    });
  })
}

constreadfile = (dir, filename) => {
  returnnewPromise((res, rej) => {
    fs.readFile(dir + filename, 'utf8', function (err, data) {
      if (err) {
        returnrej(err);
      }
      returnres(data)
    })
  })
}

Here is the output of the Lambda function:

And here are the complete CloudWatch logs:

My file contains nothing but a 'hello' inside it. You can see it on the logs.

Do keep in mind that, in Lambda Functions, you have a 512MB limit when downloading anything to /tmp. You can see the limits in the docs