2.2.6. SFTP Connector ¶

The SFTP connector supports moving data between a Secure File Transfer Protocol (SFTP) server and other supported Sqoop2 connectors.

Currently only the TO direction is supported to write records to an SFTP server. A FROM connector is pending (SQOOP-2218).

Contents

SFTP Connector
- Usage
  - Link Configuration
  - TO Job Configuration
- Loader

2.2.6.1. Usage ¶

Before executing a Sqoop2 job with the SFTP connector, set mapreduce.task.classpath.user.precedence to true in the Hadoop cluster config, for example:

<property>
  <name>mapreduce.task.classpath.user.precedence</name>
  <value>true</value>
</property>

This is required since the SFTP connector uses the JSch library (http://www.jcraft.com/jsch/) to provide SFTP functionality. Unfortunately Hadoop currently ships with an earlier version of this library which causes an issue with some SFTP servers. Setting this property ensures that the current version of the library packaged with this connector will appear first in the classpath.

To use the SFTP Connector, create a link for the connector and a job that uses the link.

2.2.6.1.1. Link Configuration¶

Inputs associated with the link configuration include:

Input	Type	Description	Example
SFTP server hostname	String	Hostname for the SFTP server. Required.	sftp.example.com
SFTP server port	Integer	Port number for the SFTP server. Defaults to 22. Optional.	2220
Username	String	The username to provide when connecting to the SFTP server. Required.	sqoop
Password	String	The password to provide when connecting to the SFTP server. Required	sqoop

2.2.6.1.1.1. Notes¶

The SFTP connector will attempt to connect to the SFTP server as part of the link validation process. If for some reason a connection can not be established, you’ll see a corresponding error message.
Note that during connection, the SFTP connector explictly disables StrictHostKeyChecking to avoid “UnknownHostKey” errors.

2.2.6.1.2. TO Job Configuration¶

Inputs associated with the Job configuration for the TO direction include:

Input	Type	Description	Example
Output directory	String	The location on the SFTP server that the connector will write files to. Required	uploads

2.2.6.1.2.1. Notes¶

The output directory value needs to be an existing directory on the SFTP server.

2.2.6.2. Loader ¶

During the loading phase, the connector will create uniquely named files in the output directory for each partition of data received from the FROM connector.

2.2.6. SFTP Connector¶

2.2.6.1. Usage¶