Sqoop 2 provides command line client that is capable of communicating with Sqoop 2 server using REST interface. Client is able to run in two modes - interactive and batch mode. Commands create, update and clone are not currently supported in batch mode. Interactive mode supports all available commands.
You can start Sqoop 2 client in interactive mode using provided sqoop.sh shell script by passing argument client:
Batch mode can be started by adding additional argument representing path to your Sqoop client script:
sqoop.sh client /path/to/your/script.sqoop
Sqoop client script is expected to contain valid Sqoop client commands, empty lines and lines starting with # that are denoting comment lines. Comments and empty lines are ignored, all other lines are interpreted. Example script:
# Specify company server set server --host sqoop2.company.net # Executing given job submission start --jid 1
Table of Contents
- Command Line Client
- Resource file
- Auxiliary Commands
- Set Command
- Show Command
- Create Command
- Update Command
- Delete Command
- Clone Command
- Submission Command
Sqoop 2 client have ability to load resource files similarly as other command line tools. At the beginning of execution Sqoop client will check existence of file .sqoop2rc in home directory of currently logged user. If such file exists, it will be interpreted before any additional actions. This file is loaded in both interactive and batch mode. It can be used to execute any batch compatible commands.
Example resource file:
# Configure our Sqoop 2 server automatically set server --host sqoop2.company.net # Run in verbose mode by default set option --name verbose --value true
Sqoop 2 contains several commands that will be documented in this section. Each command have one more functions that are accepting various arguments. Not all commands are supported in both interactive and batch mode.
Auxiliary commands are commands that are improving user experience and are running purely on client side. Thus they do not need working connection to the server.
- exit Exit client immediately. This command can be also executed by sending EOT (end of transmission) character. It’s CTRL+D on most common Linux shells like Bash or Zsh.
- history Print out command history. Please note that Sqoop client is saving history from previous executions and thus you might see commands that you’ve executed in previous runs.
- help Show all available commands with short in-shell documentation.
Set command allows to set various properties of the client. Similarly as auxiliary commands, set do not require connection to Sqoop server. Set commands is not used to reconfigure Sqoop server.
|server||Set connection configuration for server|
|option||Set various client side options|
Configure connection to Sqoop server - host port and web application name. Available arguments:
|-h, --host||localhost||Server name (FQDN) where Sqoop server is running|
|-p, --port||8080||TCP Port|
|-w, --webapp||sqoop||Tomcat’s web application name|
set server --host sqoop2.company.net --port 80 --webapp sqoop
Configure Sqoop client related options. This function have two required arguments name and value. Name represents internal property name and value holds new value that should be set. List of available option names follows:
|Option name||Default value||Description|
|verbose||false||Client will print additional information if verbose mode is enabled|
set option --name verbose --value true
Show commands displays various information including server and protocol versions or all stored meta data.
|server||Display connection information to the server (host, port, webapp)|
|version||Show version of both client and server (build numbers, supported protocols)|
|connector||Show connector meta data - set of parameters that connectors needs to create connections and jobs|
|framework||Show framework meta data - set of parameters that Sqoop framework needs to create connections and jobs|
|connection||Show created connection meta data objects|
|job||Show created job meta data objects|
Show details about configuration connection to Sqoop server.
|-a, --all||Show all connection related information (host, port, webapp)|
|-h, --host||Show host|
|-p, --port||Show port|
|-w, --webapp||Show web application name|
show server --all
Show versions of both client and server as well as supported protocols.
|-a, --all||Show all versions (server, client, protocols)|
|-c, --client||Show client version|
|-s, --server||Show server version|
|-p, --protocol||Show protocol support on client or server|
show version --all
Show connector meta data - parameters that connectors need in order to create new connection and job objects.
|-a, --all||Show information for all connectors|
|-c, --cid <x>||Show information for connector with id <x>|
show connector --all
Show framework meta data - parameters that Sqoop framework need in order to create new connection and job objects.
This function do not have any extra arguments.
Show persisted connection objects.
|-a, --all||Show all available connections from all connectors|
|-x, --xid <x>||Show connection with id <x>|
show connection --all
Creates new connection and job objects. This command is supported only in interactive mode. It will query user for all parameters that are required by specific connector and framework and persist them in Sqoop server for later use.
|connection||Create new connection object|
|job||Create new job object|
Create new connection object.
|-c, --cid <x>||Create new connection object for connector with id <x>|
create connection --cid 1
Update commands allows you to edit connection and job objects - change persisted meta data. This command is supported only in interactive mode.
Update existing connection object.
|-x, --xid <x>||Update existing connection with id <x>|
update connection --xid 1
Deletes connection and job objects from Sqoop server.
Delete existing connection object.
|-x, --xid <x>||Delete connection object with id <x>|
delete connection --xid 1
Clone command will load existing connection or job object from Sqoop server and allow user in place changes that will result in creation of new connection or job object. This command is not supported in batch mode.
Clone existing connection object.
|-x, --xid <x>||Clone connection object with id <x>|
clone connection --xid 1
Submission command is entry point for executing actual data transfers. It allows you to start, stop and retrieve status of currently running jobs.
|stop||Interrupt running job|
|status||Retrieve status for given job|
Start job (submit new submission). Starting already running job is considered as invalid operation.
|-j, --jid <x>||Start job with id <x>|
submission start --jid 1
Interrupt running job.
|-j, --jid <x>||Interrupt running job with id <x>|
submission stop --jid 1