3.6. Repository

This repository contains additional information regarding Sqoop.

3.6.1. Sqoop Schema

The DDL queries that create the Sqoop repository schema in Derby database create the following tables:

3.6.1.1. SQ_SYSTEM

Store for various state information

SQ_SYSTEM
SQM_ID: BIGINT PK
SQM_KEY: VARCHAR(64)
SQM_VALUE: VARCHAR(64)

3.6.1.2. SQ_DIRECTION

Directions

SQ_DIRECTION  
SQD_ID: BIGINT PK AUTO-GEN  
SQD_NAME: VARCHAR(64) “FROM”|”TO”

3.6.1.3. SQ_CONFIGURABLE

Configurable registration

SQ_CONFIGURABLE  
SQC_ID: BIGINT PK AUTO-GEN  
SQC_NAME: VARCHAR(64)  
SQC_CLASS: VARCHAR(255)  
SQC_TYPE: VARCHAR(32) “CONNECTOR”|”DRIVER”
SQC_VERSION: VARCHAR(64)  

3.6.1.4. SQ_CONNECTOR_DIRECTIONS

Connector directions

SQ_CONNECTOR_DIRECTIONS  
SQCD_ID: BIGINT PK AUTO-GEN  
SQCD_CONNECTOR: BIGINT FK SQCD_CONNECTOR(SQC_ID)
SQCD_DIRECTION: BIGINT FK SQCD_DIRECTION(SQD_ID)

3.6.1.5. SQ_CONFIG

Config details

SQ_CONFIG  
SQ_CFG_ID: BIGINT PK AUTO-GEN  
SQ_CFG_CONNECTOR: BIGINT FK SQ_CFG_CONNECTOR(SQC_ID), NULL for driver
SQ_CFG_NAME: VARCHAR(64)  
SQ_CFG_TYPE: VARCHAR(32) “LINK”|”JOB”
SQ_CFG_INDEX: SMALLINT  

3.6.1.6. SQ_CONFIG_DIRECTIONS

Connector directions

SQ_CONNECTOR_DIRECTIONS  
SQCD_ID: BIGINT PK AUTO-GEN  
SQCD_CONFIG: BIGINT FK SQCD_CONFIG(SQ_CFG_ID)
SQCD_DIRECTION: BIGINT FK SQCD_DIRECTION(SQD_ID)

3.6.1.7. SQ_INPUT

Input details

SQ_INPUT  
SQI_ID: BIGINT PK AUTO-GEN  
SQI_NAME: VARCHAR(64)  
SQI_CONFIG: BIGINT FK SQ_CONFIG(SQ_CFG_ID)
SQI_INDEX: SMALLINT  
SQI_TYPE: VARCHAR(32) “STRING”|”MAP”
SQI_STRMASK: BOOLEAN  
SQI_STRLENGTH: SMALLINT  
SQI_ENUMVALS: VARCHAR(100)  

3.6.1.9. SQ_JOB

Stored jobs

SQ_JOB  
SQB_ID: BIGINT PK AUTO-GEN  
SQB_NAME: VARCHAR(64)  
SQB_FROM_LINK: BIGINT FK SQ_LINK(SQ_LNK_ID)
SQB_TO_LINK: BIGINT FK SQ_LINK(SQ_LNK_ID)
SQB_CREATION_USER: VARCHAR(32)  
SQB_CREATION_DATE: TIMESTAMP  
SQB_UPDATE_USER: VARCHAR(32)  
SQB_UPDATE_DATE: TIMESTAMP  
SQB_ENABLED: BOOLEAN  

3.6.1.11. SQ_JOB_INPUT

N:M relationship job and input

SQ_JOB_INPUT  
SQBI_JOB: BIGINT PK FK SQ_JOB(SQB_ID)
SQBI_INPUT: BIGINT PK FK SQ_INPUT(SQI_ID)
SQBI_VALUE: LONG VARCHAR  

3.6.1.12. SQ_SUBMISSION

List of submissions

SQ_JOB_SUBMISSION  
SQS_ID: BIGINT PK  
SQS_JOB: BIGINT FK SQ_JOB(SQB_ID)
SQS_STATUS: VARCHAR(20)  
SQS_CREATION_USER: VARCHAR(32)  
SQS_CREATION_DATE: TIMESTAMP  
SQS_UPDATE_USER: VARCHAR(32)  
SQS_UPDATE_DATE: TIMESTAMP  
SQS_EXTERNAL_ID: VARCHAR(50)  
SQS_EXTERNAL_LINK: VARCHAR(150)  
SQS_EXCEPTION: VARCHAR(150)  
SQS_EXCEPTION_TRACE: VARCHAR(750)  

3.6.1.13. SQ_COUNTER_GROUP

List of counter groups

SQ_COUNTER_GROUP
SQG_ID: BIGINT PK
SQG_NAME: VARCHAR(75)

3.6.1.14. SQ_COUNTER

List of counters

SQ_COUNTER
SQR_ID: BIGINT PK
SQR_NAME: VARCHAR(75)

3.6.1.15. SQ_COUNTER_SUBMISSION

N:M Relationship

SQ_COUNTER_SUBMISSION  
SQRS_GROUP: BIGINT PK FK SQ_COUNTER_GROUP(SQR_ID)
SQRS_COUNTER: BIGINT PK FK SQ_COUNTER(SQR_ID)
SQRS_SUBMISSION: BIGINT PK FK SQ_SUBMISSION(SQS_ID)
SQRS_VALUE: BIGINT