控制台
华北一（售罄） 华北二
华北三
华北一（售罄） 华北二 华北三
实名认证
中国站
备案

 

免费注册
帮助文档（华北一、二）
    当前位置:  浪潮云服务官网>>文档中心>>


  • 1.1 创建mysql数据

    Mysql数据源以RDS为例。

    连接Mysql,并创建数据库sqoop和表sqoop。

     

    [root@uhadoop-******-master2 ~]# mysql -h10.13.124.35 -uroot -p'inspur' sqoop

    mysql> create database sqoop;

    Query OK, 1 row affected (0.00 sec)

    mysql> use sqoop

    Database changed

    mysql> CREATE TABLE `sqoop` (

        ->   `id` varchar(64) DEFAULT NULL,

        ->   `value` varchar(64) NOT NULL DEFAULT '',

    ->     PRIMARY KEY (`id`)

        -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

    Query OK, 0 rows affected (0.01 sec)

    mysql> insert into sqoop values ('1', 'hha'),('2', 'zhang'),('3','hehe');

    Query OK, 3 rows affected (0.00 sec)

    Records: 3  Duplicates: 0  Warnings: 0

    1.2 启动Sqoop2 CLI

    uhadoop-******-master2节点上执行。

     /home/hadoop/sqoop2/bin/sqoop.sh client

    help可查到基本使用命令

     

    sqoop:000> help

    For information about Sqoop, visit: http://sqoop.apache.org/

    Available commands:

      exit    (\x  ) Exit the shell

      history (\H  ) Display, manage and recall edit-line history

      help    (\h  ) Display this help message

      set     (\st ) Configure various client options and settings

      show    (\sh ) Display various objects and configuration options

      create  (\cr ) Create new object in Sqoop repository

      delete  (\d  ) Delete existing object in Sqoop repository

      update  (\up ) Update objects in Sqoop repository

      clone   (\cl ) Create new object based on existing one

      start   (\sta) Start job

      stop    (\stp) Stop job

      status  (\stu) Display status of a job

      enable  (\en ) Enable object in Sqoop repository

      disable (\di ) Disable object in Sqoop repository

    For help on a specific command type: help command

    1.3 查看connector

     sqoop:000> show connector

    1.4 创建mysql-link

     

    sqoop:000> create link -c 4

    Creating link for connector with id 4

    Please fill following values to create new link object

    Name: mysql-link

    Link configuration

    JDBC Driver Class: com.mysql.jdbc.Driver

    JDBC Connection String: jdbc:mysql://10.13.124.35/sqoop

    Username: root

    Password: **********

    JDBC Connection Properties:

    There are currently 0 values in the map:

    entry# protocol=tcp

    There are currently 1 values in the map:

    protocol = tcp

    entry#

    New link was successfully created with validation status OK and persistent id 1

    1.5 创建hdfs-link

     

    sqoop:000> create link -c 3

    Creating link for connector with id 3

    Please fill following values to create new link object

    Name: hdfs-link

    Link configuration

    HDFS URI: hdfs://Ucluster/

    New link was successfully created with validation status OK and persistent id 2

    1.6 显示link

     sqoop:000> show link

    1.7 创建job

     

    sqoop:000> create job -f 1 -t 2

    Creating job for links with from id 1 and to id 2

    Please fill following values to create new job object

    Name: job-mysql-to-hdfs

    From database configuration

    Schema name: sqoop

    Table name: sqoop

    Table SQL statement:

    Table column names:

    Partition column name: id

    Null value allowed for the partition column:

    Boundary query:

    ToJob configuration

    Override null value:

    Null value:

    Output format:

      0 : TEXT_FILE

      1 : SEQUENCE_FILE

    Choose: 0

    Compression format:

      0 : NONE

      1 : DEFAULT

      2 : DEFLATE

      3 : GZIP

      4 : BZIP2

      5 : LZO

      6 : LZ4

      7 : SNAPPY

      8 : CUSTOM

    Choose: 0

    Custom compression format:

    Output directory: /tmp/sqoop

    Throttling resources

    Extractors: 1

    Loaders: 1

    New job was successfully created with validation status OK  and persistent id 1

    创建完job如下

     sqoop:000> show job

    1.8 启动job

     
    sqoop:000> start job -j 1 
    Submission details 
    Job ID: 1 
    Server URL: http://localhost:12000/sqoop/ 
    Created by: root 
    Creation date: 2016-12-22 14:40:10 CST 
    Lastly updated by: root 
    External ID: job_1481968387780_0011     
    http://uhadoop-penomi-
    master1:23188/proxy/application_1481968387780_0011/ 
    2016-12-22 14:40:10 CST: BOOTING  - Progress is not available

文档是否已解决您的问题？

  已解决   未解决

如您有其它疑问，您也可以与我们技术专家联系探讨。

联系技术专家