Table of contents
No headings in the article.
Here we will perform the single and multiple file upload flow via node js to sftp server.
Note: This blog is created under the assumption that users have a basic understanding of node server creation with get/post API creation.
Let’s create a node js API –which will consume files as input and upload it on the node server and then further on sftp server.
This API will be called by the frontend application with input payload as single/multiple files.
Step 1: Create a basic node server and test API.
Initiate node server: node init –y
Here it will create a package.json file in the project directory.
Install basic packages: node install express nodemon ssh2-sftp-client multer.
Create express server and file upload API.
The basic index.js file should look like below:
Step 2: Upload single/multiple files on the node server.
- Here we can use the npm multer library which will be helpful to upload files on the node server.
Create function storage via multer.diskstorage.
This function is basically to define the output node server location in which uploaded files will be present.
Also what should be the name of the uploaded file? We can customize the name or keep the original file name. In the below example, we have tried to keep the original file name.
- Pass the above created storage multer diskstorage object with storage variable Inside multer.
- Create another multiUpload variable – which will work as middleware while calling an post api call. At the same time this multiUpload variable defines the maximum number of files can be uploaded at a time. The name is given as file. Same name need to provide as a variable name to the file while calling API.
- Pass the above multiUpload function as middleware into post api call as below:
- After calling an api /upload/MultiFile, Files will get uploaded on the node server in the specified output location. In console we can see the details of the uploaded files:
Step 3: Upload files from Node server to sftp server:
To upload files to sftp server- we need to use ssh2-sftp-client, es6-promise-pool libraries.
ssh2-sftp-client – Gives sftp functions which can help us to fetch and upload files from sftp location.
es6-promise-pool helps us to run upload task with number of concurrent threads. Let check with below example:
We need to define the PromisePool as below:
Here we are initiating pool variable to run another function called – sendFileProducer. This will run sendFileProducer function with 3 concurrent threads at a time. The thread count we can define as shown above.
Let’s first store the uploaded files details in files variable:
In the above code snippet we have defined the sendFileProducer function- which will be initiated by each thread.
The important point to note here is- As we are initiating 3 threads and – each thread will try to run above function individually. If we are uploading 2 files. Then first 2 threads will pick up those files and proceed ahead further. And as soon as 3rd thread will try to run this. Then if condition will fail and return null. Means 3rd thread will stop in this case.
This sendFileProducer function internally calling to sendFile function- in which we can include the actual code snippet to upload file on sftp server.
sendFile function defined above return the Promise – Which first of all create new sftp client with
let sftp = new sftpClient();
This sftp client need to define via installed library - ssh2-sftp-client (As shown below)
After initiating sftp client – connect sftp server with sftp.connect method. Here to connect sftp client we have to provide sftp server config details. We can create config variable as below before providing to sftp connect method.
Once sftp connection is successful- Called the sftp.put method to upload file on sftp server as below:
Here in above function we need to provide the location from where file is being uploaded and sftp location where you expects files to be uploaded.
Once the file upload is successful close the sftp connection and catch error if any
