Filebeat带来下面的变化：
- 对配置文件格式进行了重组，从JSON转换为YAML。
- 对存储当前读取文件的状态的registry file被改变。
- 命令行选项被删除并移到配置文件中。
- 输出的配置选项从libbeat继承。
- Logstash必须使用一种新的输入插件。
迁移策略
logstash
Logstash 需要安装一个新的输入插件 logstash-input-beats。在Logstash 1.5.x版本和2.x版本，该插件可以与 Logstash-Forwarder 所使用的插件logstash-input-lumberjack 并行加载。
如果你有大量的logstash-forwarder迁移到Filebeat，建议同时加载这两个插件，将其设置为不同的端口。当所有的迁移到Filebeat，即可删除Lumberjack插件。
Registry File
Registry File存储了Filbeat最后一次读的位置和状态。在Logstash-Forwarder被称为.logstash-fowarder(位于/var/lib/logstash-forwarder/.logstash-forwarder)。对于Filebeat需要将其重命名为 .filebeat。
迁移配置文件
files部分
Logstash-Forwarder 配置文件中的”files”部分转变成Filebeat配置文件中的”prospectors”部分。如：
logstash-forwarder配置文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
# The list of files configurations
“files”: [
# An array of hashes. Each hash tells what paths to watch and
# what fields to annotate on events from those paths.
{
“paths”: [
“/var/log/messages”,
“/var/log/*.log”
],
# A dictionary of fields to annotate on each event.
“fields”: { “type”: “syslog” }
}, {
# A path of “-” means stdin.
“paths”: [ “-“ ],
“fields”: { “type”: “stdin” }
}, {
“paths”: [
“/var/log/apache/httpd-*.log”
],
“fields”: { “type”: “apache” }
}
]
|
相当于Filebeat配置文件中的prospectors部分：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
filebeat:
# List of prospectors to fetch data.
prospectors:
# Each – is a prospector. Below are the prospector specific configurations
–
paths:
– /var/log/messages
– “/var/log/*.log”
–
paths:
– “-“
input_type: stdin
document_type: stdin
–
paths:
– “/var/log/apache/httpd-*.log”
document_type: apache
|
引入了一个新的选项document_type，如果没有类型被定义则默认为log。如果Filebeat被直接用来Elasticsearch索引，那么当在索引时document_type决定文档类型。
network部分
Filebeat可以于Logstash直接通信，此外，Filebeat还可以直接向elasticsearch插入日志条目。
logstash-forwarder配置文件：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
# The network section covers network configuration 🙂
“network”: {
# A list of downstream servers listening for our messages.
# logstash-forwarder will pick one at random and only switch if
# the selected one appears to be dead or unresponsive
“servers”: [ “localhost:5043” ],
# The path to your client ssl certificate (optional)
“ssl certificate”: “./logstash-forwarder.crt”,
# The path to your client ssl key (optional)
“ssl key”: “./logstash-forwarder.key”,
# The path to your trusted ssl CA file. This is used
# to authenticate your downstream server.
“ssl ca”: “./logstash-forwarder.crt”,
# Network timeout in seconds. This is most important for
# logstash-forwarder determining whether to stop waiting for an
# acknowledgement from the downstream server. If an timeout is reached,
# logstash-forwarder will assume the connection or server is bad and
# will connect to a server chosen at random from the servers list.
“timeout”: 15
}
|
Filebeat相当于：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
output:
logstash:
enabled: true
# The list of downstream Logstash servers.
hosts:
– localhost:5043
tls:
# The path to your SSL client certificate.
certificate: ./logstash–forwarder.crt
# The path to your SSL client certificate key.
certificate_key: ./logstash–forwarder.key
# The path to your trusted SSL CA file. This is used
# to authenticate your downstream server.
certificate_authorities:
– ./logstash–forwarder.crt
# Network timeout in seconds.
timeout: 15
|
当定义多台主机，类似于Logstash-forwarder行为，Filebeat默认随机选择一个主机建立连接。Filebeat可用设置为负载均衡。参见：https://www.elastic.co/guide/en/beats/libbeat/1.0.0-rc1/configuration.html#loadbalance
更改后的配置文件选项
配置文件的重构，有些选项被删除或改名。下面是更改的条目列表：
| Config Option | Action |
|---|---|
deadTime |
deadTime was renamed to ignoreOlder. In case a file is not changed for ignoreOlder, the file handler will be closed. If the file is changed again after ignoreOlder has passed, it is be reopened. |
netTimeout |
netTimeout was removed as it is replaced by the Timeout option in libbeat. |
log-to-syslog andsyslog |
Both options were removed as logging is part of the libbeat config. |
完整的实例
Logstash-Forwarder配置文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
|
{
“files”: [
{
“paths”: [
“/var/log/*.log”
],
“fields”: { “type”: “syslog” }
}
],
“network”: {
“servers”: [ “localhost:5043” ],
}
}
|
Filebeat配置文件：
|
1
2
3
4
5
6
7
8
9
10
11
|
filebeat:
prospectors:
–
paths:
– “/var/log/*.log”
fields:
type: syslog
output:
elasticsearch:
enabled: true
hosts: [“http://localhost:5043”]
|
命令行选项
大部分 logstash-forwarder命令行被删除并移到配置文件中，重命名的命令行选项列表如下：
| Command Line Option | Config File Option | Description |
|---|---|---|
-config |
-c |
The config options was split up in two part. The base and required config is linked with -c. Additional config files can be linked as part of the config file. Note: Additional config files must be in a different directory than the main config file. |
-config |
config_dir |
Path to directory with additional configuration files |
-idle-timeout |
idle_timeout |
idle_timeout was moved to the config file and removed as flag. |
-spool-size |
spool_size |
spool_size was moved to the config file and removed as flag. |
-harvester-buff-size |
harvester_buffer_size |
harvester_buffer_size was moved to the config file and removed as flag. It can now be configured specific for each harvester. |
-tail |
tail_files |
tail_files was moved to the config file and removed as flag. It can now be configured specific for each prospector. |
-cpuProfileFile |
cpuProfileFile option was removed. The profiling options of libbeat can be used instead. For more details on profiling see https://github.com/elastic/libbeat/issues/122 |
|
-quiet |
The quiet option was removed. Libbeat is used for logging and the libbeat configuration options have to be used. |
其它的一些改变
- 包
一个显着的变化是registry file名称取决于包封装类型：
.tar.gz 和 .tgz 名称为.filebeat
DEB 和 RPM 名称为/usr/lib/filebeat/registry
Windows zip包 名称为c:\ProgramData\filebeat\registry - TLS默认是关闭的
- 日志
Filebeat使用libbeat日志，也可以记录到轮滚的文件，而不是系统日志。
文章转载来自：ttlsa.com
