(ElasticSearch+LogStash+Kibana)ELK搭建在Mac/Linux系统上6.4.2版本

作者: admin 分类: ELK 发布时间: 2018-11-12 23:11  阅读: 1,083 views

之前对ELK完全没有概念,工作中需要引入,所以就花时间看了下相关资料。作为开源的软件,功能配套很齐全且插件完善。很多场景可以使用elk。

监控文件日志、系统日志、报警、图形化展示数据等。以下是整体的环境安装过程以及配置,基本是在Mac上试验一边后,去阿里云ecs上做部署。有

部分区别,具体的信息都可以根据ELK各日志文件的内容进行查看分析。

 


ELK官方下载地址:https://www.elastic.co/downloads

全部环境

ElasticSearch6.4.2
Logstash6.4.2
kibana6.4.2
filebeat6.4.2
python2.7
Kibana_Hanization-master汉化插件
sentinl-v6.4.2.


 

步骤

1.安装ElasticSearch6.4.2
2.安装Logstash6.4.2(如果不用可以舍去)
3.安装filebeat6.4.2
4.安装kibana6.4.2-配置连接elasticsearch
5.用python对kibana进行汉化
5.1 ElasticSearch6.4.2 和 kibana6.4.2可以对外访问
6.启动kibana试用版本(可以使用高级功能)
7.生成证书[配置elasticsearch证书重启]
8.创建密码[配置kibana\filebeat相关账号并启动]
9.配置x-pack破解jar文件
10.重启,登录kibana上传证书 (整个破解过程完成)
11.kibana安装 sentinl插件
12.配置告警邮件信息
13.至此,基本东西都可以使用

前期:索引的建立要多注意学习,否则不方便后期数据的排序等处理


一、在MAC环境下安装ElasticSearch6.4.2

下载好ElasticSearch对应环境包之后,解压到对应目录下,用终端运行 */ElasticSearch6.4.2/bin/ElasticSearch文件

出现以下信息

Last login: Thu Nov 1 19:43:45 on ttys000
chenhailongdeMacBook-Pro:~ chenhailong$ /Users/chenhailong/Downloads/tools/elasticsearch-6.4.2/bin/elasticsearch ; exit;
[2018-11-02T19:04:01,227][INFO ][o.e.n.Node ] [] initializing ...
[2018-11-02T19:04:01,321][INFO ][o.e.e.NodeEnvironment ] [pdEpNda] using [1] data paths, mounts [[/ (/dev/disk1s1)]], net usable_space [149.6gb], net total_space [233.4gb], types [apfs]
[2018-11-02T19:04:01,321][INFO ][o.e.e.NodeEnvironment ] [pdEpNda] heap size [990.7mb], compressed ordinary object pointers [true]
[2018-11-02T19:04:01,324][INFO ][o.e.n.Node ] [pdEpNda] node name derived from node ID [pdEpNda6RxmhLf0bS58edQ]; set [node.name] to override
[2018-11-02T19:04:01,324][INFO ][o.e.n.Node ] [pdEpNda] version[6.4.2], pid[32294], build[default/tar/04711c2/2018-09-26T13:34:09.098244Z], OS[Mac OS X/10.13.3/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_171/25.171-b11]
[2018-11-02T19:04:01,325][INFO ][o.e.n.Node ] [pdEpNda] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/var/folders/ll/dms152m534b4nrtscw36q12c0000gn/T/elasticsearch.lmidN4IW, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/Users/chenhailong/Downloads/tools/elasticsearch-6.4.2, -Des.path.conf=/Users/chenhailong/Downloads/tools/elasticsearch-6.4.2/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2018-11-02T19:04:03,921][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [aggs-matrix-stats]
[2018-11-02T19:04:03,921][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [analysis-common]
[2018-11-02T19:04:03,922][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [ingest-common]
[2018-11-02T19:04:03,922][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [lang-expression]
[2018-11-02T19:04:03,922][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [lang-mustache]
[2018-11-02T19:04:03,922][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [lang-painless]
[2018-11-02T19:04:03,922][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [mapper-extras]
[2018-11-02T19:04:03,923][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [parent-join]
[2018-11-02T19:04:03,923][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [percolator]
[2018-11-02T19:04:03,923][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [rank-eval]
[2018-11-02T19:04:03,924][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [reindex]
[2018-11-02T19:04:03,924][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [repository-url]
[2018-11-02T19:04:03,925][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [transport-netty4]
[2018-11-02T19:04:03,925][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [tribe]
[2018-11-02T19:04:03,925][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-core]
[2018-11-02T19:04:03,925][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-deprecation]
[2018-11-02T19:04:03,925][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-graph]
[2018-11-02T19:04:03,925][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-logstash]
[2018-11-02T19:04:03,925][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-ml]
[2018-11-02T19:04:03,926][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-monitoring]
[2018-11-02T19:04:03,926][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-rollup]
[2018-11-02T19:04:03,926][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-security]
[2018-11-02T19:04:03,926][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-sql]
[2018-11-02T19:04:03,926][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-upgrade]
[2018-11-02T19:04:03,927][INFO ][o.e.p.PluginsService ] [pdEpNda] loaded module [x-pack-watcher]
[2018-11-02T19:04:03,927][INFO ][o.e.p.PluginsService ] [pdEpNda] no plugins loaded
[2018-11-02T19:04:08,476][INFO ][o.e.x.s.a.s.FileRolesStore] [pdEpNda] parsed [0] roles from file [/Users/chenhailong/Downloads/tools/elasticsearch-6.4.2/config/roles.yml]
[2018-11-02T19:04:09,134][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/32312] [Main.cc@109] controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
[2018-11-02T19:04:09,662][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-11-02T19:04:09,891][INFO ][o.e.d.DiscoveryModule ] [pdEpNda] using discovery type [zen]
[2018-11-02T19:04:10,717][INFO ][o.e.n.Node ] [pdEpNda] initialized
[2018-11-02T19:04:10,717][INFO ][o.e.n.Node ] [pdEpNda] starting ...
[2018-11-02T19:04:10,998][INFO ][o.e.t.TransportService ] [pdEpNda] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2018-11-02T19:04:14,238][INFO ][o.e.c.s.MasterService ] [pdEpNda] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {pdEpNda}{pdEpNda6RxmhLf0bS58edQ}{Zrue9LmCQcSjSGpTqtNnzg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=8589934592, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2018-11-02T19:04:14,243][INFO ][o.e.c.s.ClusterApplierService] [pdEpNda] new_master {pdEpNda}{pdEpNda6RxmhLf0bS58edQ}{Zrue9LmCQcSjSGpTqtNnzg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=8589934592, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {pdEpNda}{pdEpNda6RxmhLf0bS58edQ}{Zrue9LmCQcSjSGpTqtNnzg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=8589934592, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
[2018-11-02T19:04:14,269][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [pdEpNda] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2018-11-02T19:04:14,270][INFO ][o.e.n.Node ] [pdEpNda] started
[2018-11-02T19:04:14,282][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [pdEpNda] Failed to clear cache for realms [[]]
[2018-11-02T19:04:14,334][INFO ][o.e.g.GatewayService ] [pdEpNda] recovered [0] indices into cluster_state
[2018-11-02T19:04:14,509][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.watch-history-9] for index patterns [.watcher-history-9*]
[2018-11-02T19:04:14,526][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.watches] for index patterns [.watches*]
[2018-11-02T19:04:14,539][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.triggered_watches] for index patterns [.triggered_watches*]
[2018-11-02T19:04:14,562][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-6-*]
[2018-11-02T19:04:14,594][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.monitoring-es] for index patterns [.monitoring-es-6-*]
[2018-11-02T19:04:14,619][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.monitoring-beats] for index patterns [.monitoring-beats-6-*]
[2018-11-02T19:04:14,637][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.monitoring-alerts] for index patterns [.monitoring-alerts-6]
[2018-11-02T19:04:14,661][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pdEpNda] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-6-*]
[2018-11-02T19:04:14,772][INFO ][o.e.l.LicenseService ] [pdEpNda] license [eae58a03-89f0-4725-9982-a56f44c428fc] mode [basic] - valid

 

在linux下, 可以执行 ./bin/elasticsearch -d 会在后台启动, ctrl+x的时候不会关闭

打开浏览器访问127.0.0.1:9200 说明成功


二、在MAC环境下安装LogStash6.4.2

下载好LogStash6.4.2对包之后,解压到对应目录下,要先在bin目录下创建文件 logstash-simple.conf 并保存以下内容

 

input { stdin { } }
output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

然后在黑屏运行 ./bin/logstash -f logstash-sample.conf

会出现以下信息

Sending Logstash logs to /Users/chenhailong/Downloads/tools/logstash-6.4.2/logs which is now configured via log4j2.properties
[2018-11-02T19:56:53,461][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/Users/chenhailong/Downloads/tools/logstash-6.4.2/data/queue"}
[2018-11-02T19:56:53,475][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/Users/chenhailong/Downloads/tools/logstash-6.4.2/data/dead_letter_queue"}
[2018-11-02T19:56:53,595][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-11-02T19:56:53,639][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"e969e65c-57f3-4632-a098-7a2dbab6b7ab", :path=>"/Users/chenhailong/Downloads/tools/logstash-6.4.2/data/uuid"}
[2018-11-02T19:56:54,242][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.2"}
[2018-11-02T19:56:57,448][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-11-02T19:56:57,929][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-11-02T19:56:57,938][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-11-02T19:56:58,308][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-11-02T19:56:58,384][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-11-02T19:56:58,387][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-11-02T19:56:58,414][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-11-02T19:56:58,429][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-11-02T19:56:58,445][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-11-02T19:56:58,541][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/logstash
The stdin plugin is now waiting for input:
[2018-11-02T19:56:58,576][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#"}
[2018-11-02T19:56:58,655][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-11-02T19:56:59,090][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

发现在LogStash6.4.2/data生成了文件,控制台输入文字后回车,发下也返回了格式化的数据

如下:

qwerqwerqwerqwer

{
  "message" => "qwerqwerqwerqwer",
  "@timestamp" => 2018-11-02T12:11:26.004Z,
  "@version" => "1",
  "host" => "chenhailongdeMacBook-Pro.local"
}

logstash利用grok插件切割日志


三. 在MAC环境下安装filebeat6.4.2

logstash可以日志收集、分析、过滤。部署在客户端比较重,有更轻量级的filebeat。

下载好filebeat后,解压到对应目录

对filebeat.yml配置文件进行配置【网上搜罗】

运行 ./filebeat -e -c /home/elk/filebeat/filebeat.yml可以看到以下信息

chenhailongdeMacBook-Pro:filebeat-6.4.2-darwin-x86_64 chenhailong$ ./filebeat -e -c filebeat.yml 2018-11-07T10:15:19.643+0800 INFO instance/beat.go:544 Home path: [/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64] Config path: [/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64] Data path: [/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64/data] Logs path: [/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64/logs]
2018-11-07T10:15:19.645+0800 INFO instance/beat.go:551 Beat UUID: 644df73f-9c7d-4687-83d7-1ff62d4f9aa1
2018-11-07T10:15:19.645+0800 INFO [beat] instance/beat.go:768 Beat info {"system_info": {"beat": {"path": {"config": "/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64", "data": "/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64/data", "home": "/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64", "logs": "/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64/logs"}, "type": "filebeat", "uuid": "644df73f-9c7d-4687-83d7-1ff62d4f9aa1"}}}
2018-11-07T10:15:19.645+0800 INFO [beat] instance/beat.go:777 Build info {"system_info": {"build": {"commit": "e193f6d68b25b7ddbe3a3ed8d60bc07fea1ef800", "libbeat": "6.4.2", "time": "2018-09-26T12:42:00.000Z", "version": "6.4.2"}}}
2018-11-07T10:15:19.645+0800 INFO [beat] instance/beat.go:780 Go runtime info {"system_info": {"go": {"os":"darwin","arch":"amd64","max_procs":4,"version":"go1.10.3"}}}
2018-11-07T10:15:19.646+0800 INFO [beat] instance/beat.go:784 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-10-22T18:30:50.300998+08:00","hostname":"chenhailongdeMacBook-Pro.local","ips":["127.0.0.1/8","::1/128","fe80::1/64","fe80::403:6554:cc7e:8718/64","192.168.20.140/22","fe80::94dc:1bff:fe00:bc5/64","fe80::fa54:2b42:ece4:afab/64"],"kernel_version":"17.4.0","mac_addresses":["8c:85:90:cd:a0:47","0e:85:90:cd:a0:47","96:dc:1b:00:0b:c5","72:00:25:68:ae:01","72:00:25:68:ae:00","72:00:25:68:ae:01"],"os":{"family":"darwin","platform":"darwin","name":"Mac OS X","version":"10.13.3","major":10,"minor":13,"patch":3,"build":"17D47"},"timezone":"CST","timezone_offset_sec":28800}}} 2018-11-07T10:15:19.647+0800 INFO [beat] instance/beat.go:813 Process info {"system_info": {"process": {"cwd": "/Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64", "exe": "./filebeat", "name": "filebeat", "pid": 62154, "ppid": 61954, "start_time": "2018-11-07T10:15:19.529+0800"}}}
2018-11-07T10:15:19.647+0800 INFO instance/beat.go:273 Setup Beat: filebeat; Version: 6.4.2
2018-11-07T10:15:19.648+0800 INFO elasticsearch/client.go:163 Elasticsearch url: http://localhost:9200
2018-11-07T10:15:19.657+0800 INFO pipeline/module.go:98 Beat name: chenhailongdeMacBook-Pro.local
2018-11-07T10:15:19.665+0800 INFO instance/beat.go:367 filebeat start running.
2018-11-07T10:15:19.665+0800 INFO [monitoring] log/log.go:114 Starting metrics logging every 30s
2018-11-07T10:15:19.665+0800 INFO registrar/registrar.go:97 No registry file found under: /Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64/data/registry. Creating a new registry file.
2018-11-07T10:15:19.665+0800 INFO registrar/registrar.go:134 Loading registrar data from /Users/chenhailong/Downloads/tools/filebeat-6.4.2-darwin-x86_64/data/registry
2018-11-07T10:15:19.666+0800 INFO registrar/registrar.go:141 States Loaded from registrar: 0
2018-11-07T10:15:19.666+0800 INFO crawler/crawler.go:72 Loading Inputs: 1
2018-11-07T10:15:19.666+0800 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 0
2018-11-07T10:15:19.666+0800 INFO cfgfile/reload.go:141 Config reloader started
2018-11-07T10:15:19.666+0800 INFO cfgfile/reload.go:196 Loading of config files completed.

四、在MAC环境下安装kibana6.4.2

下载好kibana6.4.2对包之后,解压后打开config/kibana.yml配置文件,去掉注释如下。

# The URL of the Elasticsearch instance to use for all your queries.

elasticsearch.url: “http://localhost:9200”

启动 bin/kibana,出现以下信息

chenhailongdeMacBook-Pro:kibana-6.4.2-darwin-x86_64 chenhailong$ ./bin/kibana
log [12:23:40.710] [info][status][plugin:kibana@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:40.749] [info][status][plugin:elasticsearch@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:40.751] [info][status][plugin:xpack_main@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:40.756] [info][status][plugin:searchprofiler@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:40.759] [info][status][plugin:ml@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:40.796] [info][status][plugin:tilemap@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:40.798] [info][status][plugin:watcher@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:40.817] [info][status][plugin:license_management@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:40.819] [info][status][plugin:index_management@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:41.002] [info][status][plugin:timelion@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.005] [info][status][plugin:graph@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:41.017] [info][status][plugin:monitoring@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.019] [warning][security] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml
log [12:23:41.021] [warning][security] Session cookies will be transmitted over insecure connections. This is not recommended.
log [12:23:41.026] [info][status][plugin:security@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:41.042] [info][status][plugin:grokdebugger@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:41.045] [info][status][plugin:dashboard_mode@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.049] [info][status][plugin:logstash@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:41.067] [info][status][plugin:apm@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.072] [info][status][plugin:console@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.074] [info][status][plugin:console_extensions@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.077] [info][status][plugin:notifications@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.080] [info][status][plugin:metrics@6.4.2] Status changed from uninitialized to green - Ready
log [12:23:41.754] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml
log [12:23:41.756] [info][status][plugin:reporting@6.4.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [12:23:41.931] [info][status][plugin:elasticsearch@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.086] [info][license][xpack] Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active
log [12:23:42.089] [info][status][plugin:xpack_main@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.089] [info][status][plugin:searchprofiler@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.090] [info][status][plugin:ml@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.090] [info][status][plugin:tilemap@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.090] [info][status][plugin:watcher@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.091] [info][status][plugin:index_management@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.091] [info][status][plugin:graph@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.093] [info][status][plugin:grokdebugger@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.093] [info][status][plugin:logstash@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.094] [info][status][plugin:reporting@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.095] [info][kibana-monitoring][monitoring-ui] Starting monitoring stats collection
log [12:23:42.097] [info][status][plugin:security@6.4.2] Status changed from yellow to green - Ready
log [12:23:42.195] [info][license][xpack] Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active
log [12:23:42.852] [info][listening][server][http] Server running at http://localhost:5601

nohup ../bin/kibana & 这样可以后台启动并产生日志记录\其他程序也适用


五. 将KIbana进行汉化

下载好python后,在电脑上进行安装、出现以下信息表示正确【2.7版本】

chenhailongdeMacBook-Pro:kibana-6.4.2-darwin-x86_64 chenhailong$ python
Python 2.7.15 (default, Jun 17 2018, 12:46:58)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

然后去网络上寻找kibana汉化包 https://blog.csdn.net/qq_28449663/article/details/79868334

解压后执行 python main.py /路径

出现以下信息表示正确【几分钟时间】

chenhailongdeMacBook-Pro:Kibana_Hanization-master chenhailong$ python main.py /Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/ml.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/timelion.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/status_page.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/monitoring.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/kibana.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/vendors.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/login.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/commons.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/optimize/bundles/apm.bundle.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/node_modules/x-pack/plugins/monitoring/ui_exports.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/node_modules/x-pack/plugins/monitoring/public/register_feature.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/ui/public/chrome/directives/global_nav/global_nav.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/ui_setting_defaults.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/netflow/on_prem.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/netflow/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/netflow/elastic_cloud.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/netflow/on_prem_elastic_cloud.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/netflow/common_instructions.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/nginx_metrics/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/osquery_logs/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/redis_metrics/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/apm/on_prem.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/apm/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/apm/apm_client_instructions.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/apm/apm_server_instructions.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/mysql_logs/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/apache_logs/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/kafka_logs/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/system_logs/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/apache_metrics/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/kubernetes_metrics/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/system_metrics/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/redis_logs/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/nginx_logs/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/docker_metrics/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/server/tutorials/mysql_metrics/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/common/tutorials/metricbeat_instructions.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/common/tutorials/filebeat_instructions.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/public/home/components/tutorial_directory.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/public/home/components/add_data.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/public/management/sections/objects/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/public/management/sections/objects/components/objects_table/components/header/header.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/public/dashboard/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kibana/translations/en.json]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/timelion/index.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kbn_vislib_vis_types/public/line.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kbn_vislib_vis_types/public/histogram.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kbn_vislib_vis_types/public/heatmap.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kbn_vislib_vis_types/public/area.js]已翻译。
文件[/Users/chenhailong/Downloads/tools/kibana-6.4.2-darwin-x86_64/src/core_plugins/kbn_vislib_vis_types/public/horizontal_bar.js]已翻译。

 

恭喜,Kibana汉化完成!

最终效果如下。 由于过程不可逆,且过程中可能对一些资源进行整理。所以在ELK的环境搭建好之后,先进行汉化后在真正使用。

 


五. KIbana、 elasticsearch 可以外网访问

默认只能本机访问,如果需要外网访问的话,需要修改配置

kibana : 打开 ./config/kibana.conf 将 server.host: “localhost” 改为 server.host: “0.0.0.0”

elasticsearch: 打开./config/elasticsearch.conf 将network.host: 192.168.0.1改为network.host: 0.0.0.0


 

 

六、启用试用版本【可以使用机器学习-监视器等会员功能】

点击系统监控 – 开启监控 – ElasticSearch模块 后的license相关 – 点击Start a 30-day trial


七. 相关证书生成并配置

ca证书生成

elasticsearch-6.4.2/bin/elasticsearch-certutil ca --ca-dn "CN=WolfBolin Elatic CA" --out /home/admin/certs/wolfbolin-elastic-ca.p12

cert证书生成

elasticsearch-6.4.2/bin/elasticsearch-certutil cert -ca /etc/elasticsearch/certs/wolfbolin-elastic-ca.p12 --out /etc/elasticsearch/certs/wolfbolin-elastic-certificates.p12

这里最好用admin账户生成,并赋予可读权限。放到其他目录在启动es时会出现 access deined异常, 这里是放到了 elasticsearch/conf/目录下

对elasticsearch.yml进行配置,在最下方添加,保存后重启

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: wolfbolin-elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: wolfbolin-elastic-certificates.p12

 


 

八、创建账号密码

在目录下执行 elasticsearch-6.4.2/bin/elasticsearch-setup-passwords interactive 【手动创建密码】

Initiating the setup of passwords for reserved users elastic,kibana,logstash_system.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [elastic]

在启用证书后,访问kibana、elasticsearch都需要认证了。在各个配置文件添加账号密码

1. kibana.yml

找到以下地方去掉账号密码的#号,并输入正确的用户名、密码

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
elasticsearch.username: "elastic"
elasticsearch.password: "123456"

2. filebeat.yml

找到以下地方去掉账号密码的#号,并输入正确的用户名、密码【注意格式对齐】

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
host: "xxx.xx.xxx.xx:5601"
username: "kibana"
password: "123456"

还有这个地方

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["xxx.xx.xxx.xx:9200"]
# Optional protocol and basic auth credentials.
#protocol: "https"
username: "elastic"
password: "123456"

都需要重启以后生效,发现可以正常发送监控日志数据/登录Kibana

 


九、破解x-pack-core-6.4.2.jar包

先备份

在目录创建 LicenseVerifier.java文件 、然后创建 XPackBuild.java ,具体查看

https://blog.csdn.net/qq_36731677/article/details/83090036 的处理过程

 

 


十、登录kibana上传最新的license证书

这个最好在官网先获取一份license证书的邮件,拿到正确的json证书格式文件

对应修改

修改type字段为platinum,表示铂金版

修改expiry_date_in_millis字段为2147482800000,表示时间尽头

修改max_nodes字段为1000,表示集群数量

保存后进行上传,在系统监控 – elasticsearch 后发现以下信息说明成功

Platinum license will expire on January 19,2038

 

 

猜你想看:

环境搭建之linux下ELK搭建好之后配置sentinl插件,进行邮件告警

ELK环境搭建之filebeat根据不同的监控日志文件建立不同的索引

Java连接ElasticSearch(low-level-rest-client)的配置和数据的增加/查询


   原创文章,转载请标明本文链接: (ElasticSearch+LogStash+Kibana)ELK搭建在Mac/Linux系统上6.4.2版本

如果觉得我的文章对您有用,请随意打赏。您的支持将鼓励我继续创作!

4条评论
  • Car for Kids in Dubai

    2019年3月28日 21:55

    Excellent blog here! Also your website loads up very fast!

    What web host are you using? Can I get your affiliate link to your host?
    I wish my web site loaded up as fast as yours lol

发表评论

电子邮件地址不会被公开。 必填项已用*标注

更多阅读