logstash配置jdbc

    ELK   ELK logstash

安装jdbc插件

### 当前版本logstash5.5.1

./bin/logstash-plugin install logstash-input-jdbc

配置jdbc

  • 下载jdbc_driver_library mysql-connector-java-5.1.36.jar
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.36/mysql-connector-java-5.1.36.jar
  • 配置logstash

### vi jdbc-test.conf
input {
    stdin {
    }
    jdbc {
        # mysql链接地址
        jdbc_connection_string => "jdbc:mysql://localhost:3306/test"
        # 用户、密码
        jdbc_user => "root"
        jdbc_password => ""
        # jdbc driver路径
        jdbc_driver_library => "path/to/mysql-connector-java-5.1.36.jar"
        # the name of the driver class for mysql
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_paging_enabled => "true"
        jdbc_page_size => "50000"
        ## 执行的sql 文件路径+名称 statement => "sql语句"
        statement_filepath => "./sql/jdbc.sql"
        ## schedule：设置监听间隔
        schedule => "* * * * *"
        type => "jdbc"
    }
}

filter {
    json {
        source => "message"
        remove_field => ["message"]
    }
}

output {
    elasticsearch {
        hosts => ["http://localhost:9200"]
        index => "jdbc-test-%{+YYYY.MM.dd}"
    }
    stdout{
        codec => rubydebug
    }
}

运行测试

../bin/logstash -f ./jdbc-test.conf

参考文档