Telegraf Syslog 输入插件实战从 RFC5424/RFC3164 解析到 Rsyslog 对接的完整配置指南【免费下载链接】telegrafAgent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.项目地址: https://gitcode.com/GitHub_Trending/te/telegrafTelegraf 的inputs.syslog插件是一个服务类输入插件它监听通过 Unix Domain Socket、UDP、TCP 或 TLS 传输的 syslog 消息支持带或不带 octet counting 分帧将符合 RFC5424 或 BSD syslogRFC3164格式的日志消息解析为带丰富标签与字段的指标。读完本文你将掌握该插件的完整参数配置、分帧/传输协议的选择依据、与 Rsyslog 的对接方法以及底层源码中解析器与指标映射的实际实现逻辑。插件定位为什么它是 Service Inputinputs.syslog在 插件源码 中通过inputs.Add(syslog, ...)注册其Gather()方法直接返回 nil见 syslog.go#L146-L148——指标的产生不依赖采集周期而来自监听套接字上到达的数据。这带来两个与其他输入插件的关键差异全局或插件级的interval设置可能不生效--test、--test-wait、--once等 CLI 选项对该插件可能不会产生输出。从源码结构看插件在Start()中调用socket.Setup()建立监听随后按协议族分发到两条处理路径流式套接字tcp/tcp4/tcp6/unix/unixpacket走createStreamDataHandler数据报套接字udp/udp4/udp6/ip/ip4/ip6/unixgram走createDatagramDataHandler见 syslog.go#L117-L144。完整配置参考以下配置完整继承自 plugins/inputs/syslog/README.md 与 sample.conf所有选项均与源码中Syslog结构体自身字段 内嵌的 socket.Config一一对应[[inputs.syslog]] ## Protocol, address and port to host the syslog receiver. ## If no host is specified, then localhost is used. ## If no port is specified, 6514 is used (RFC5425#section-4.1). ## ex: server tcp://localhost:6514 ## server udp://:6514 ## server unix:///var/run/telegraf-syslog.sock ## When using tcp, consider using tcp4 or tcp6 to force the usage of IPv4 ## or IPV6 respectively. There are cases, where when not specified, a system ## may force an IPv4 mapped IPv6 address. server tcp://127.0.0.1:6514 ## Permission for unix sockets (only available on unix sockets) ## This setting may not be respected by some platforms. To safely restrict ## permissions it is recommended to place the socket into a previously ## created directory with the desired permissions. ## ex: socket_mode 777 # socket_mode ## Maximum number of concurrent connections (only available on stream sockets like TCP) ## Zero means unlimited. # max_connections 0 ## Read timeout (only available on stream sockets like TCP) ## Zero means unlimited. # read_timeout 0s ## Optional TLS configuration (only available on stream sockets like TCP) # tls_cert /etc/telegraf/cert.pem # tls_key /etc/telegraf/key.pem ## Enables client authentication if set. # tls_allowed_cacerts [/etc/telegraf/clientca.pem] ## Maximum socket buffer size (in bytes when no unit specified) ## For stream sockets, once the buffer fills up, the sender will start ## backing up. For datagram sockets, once the buffer fills up, metrics will ## start dropping. Defaults to the OS default. # read_buffer_size 64KiB ## Period between keep alive probes (only applies to TCP sockets) ## Zero disables keep alive probes. Defaults to the OS configuration. # keep_alive_period 5m ## Content encoding for message payloads ## Can be set to gzip for compressed payloads or identity for no encoding. # content_encoding identity ## Maximum size of decoded packet (in bytes when no unit specified) # max_decompression_size 500MB ## List of allowed source IP addresses for incoming packets/messages. ## If not specified or empty, all sources are allowed. # allowed_sources [] ## Source IP for Source-Specific Multicast (SSM / IGMPv3) # multicast_source ## Framing technique used for messages transport ## Available settings are: ## octet-counting -- see RFC5425#section-4.3.1 and RFC6587#section-3.4.1 ## non-transparent -- see RFC6587#section-3.4.2 # framing octet-counting ## The trailer to be expected in case of non-transparent framing (default LF). ## Must be one of LF, or NUL. # trailer LF ## Whether to parse in best effort mode or not (default false). ## By default best effort parsing is off. # best_effort false ## The RFC standard to use for message parsing ## By default RFC5424 is used. RFC3164 only supports UDP transport (no streaming support) ## Must be one of RFC5424, or RFC3164. # syslog_standard RFC5424 ## Character to prepend to SD-PARAMs (default _). ## A syslog message can contain multiple parameters and multiple identifiers within structured data section. ## Eg., [id1 name1val1 name2val2][id2 name1val1 nameAvalA] ## For each combination a field is created. ## Its name is created concatenating identifier, sdparam_separator, and parameter name. # sdparam_separator _ ## Maximum length allowed for a single message (in bytes when no unit specified) ## Only applies to octet-counting framing. # max_message_length 8KiB插件还支持额外的全局与插件级配置修改指标、标签、字段、创建别名、插件排序等详见 CONFIGURATION.md。地址与默认值Init 阶段的校验逻辑配置解析与校验集中在Syslog.Init()见 syslog.go#L55-L115源码确认了若干关键行为协议必填server地址必须包含://否则返回missing protocol within address错误支持的协议族流式为tcp、tcp4、tcp6、unix、unixpacket数据报为udp、udp4、udp6、ip、ip4、ip6、unixgram其他值报unknown protocol默认地址未配置server时回退为tcp://127.0.0.1:6514仅指定主机时自动补默认端口6514测试用例 TestAddressDefault / TestAddressDefaultPort 验证了这两点默认值填充framing缺省为octet-countingsyslog_standard缺省为RFC5424sdparam_separator缺省为_trailer在插件注册时初始化为LF见 syslog.go#L358-L364。这些校验行为均有对应测试TestAddressMissingProtocol、TestAddressUnknownProtocol、TestMessageSizeCustom等见 syslog_test.go。套接字层选项的底层实现socket_mode、max_connections、read_timeout、keep_alive_period、content_encoding、max_decompression_size、allowed_sources、multicast_source、read_buffer_size等选项并非Syslog自身字段而是来自内嵌的 socket.Config——该通用套接字层同时服务于多个监听类插件。源码中的几个细节值得注意数据报默认缓冲当read_buffer_size未显式设置时UDP/unixgram 监听器将读取缓冲设为64 * 1024字节datagram.go#L192-L194 及 #L241、#L252注释标明这是按 IP 报文大小取的上限来源白名单allowed_sources在数据报接收datagram.go#L79与流式连接接受stream.go#L263两处都会执行isSourceAllowed检查即该选项对流式与数据报传输同时生效读超时告警对 TCP 流式套接字设置read_timeout 0时Init()会输出Read timeout set! Connections, inactive for the set duration, will be closed!警告syslog.go#L99-L101测试TestReadTimeoutWarning断言了该日志gzip 内容编码content_encoding gzip通过internal.WithMaxDecompressionSize等选项在接收层解压datagram.go#L260-L261配合max_decompression_size限制解压后的最大尺寸防范解压炸弹。消息分帧Framing流式传输的核心概念framing选项仅对流式传输TCP/Unix 流套接字生效决定消息在字节流中如何被切分默认值为octet-counting取值含义规范出处octet-counting默认每条消息前缀数字字节长度接收端按计数读取RFC5425#section-4.3.1、RFC6587#section-3.4.1non-transparent以固定尾标trailer结束一条消息RFC6587#section-3.4.2trailer仅在framing non-transparent时有效必须取LF默认或NUL之一。在源码中非透明分帧通过nontransparent.WithTrailer(s.Trailer)传递给解析器syslog.go#L175-L177而max_message_length仅在 octet-counting 分帧下有意义——源码注释明确说明 go-syslog 在非透明分帧下将其视为 no-opsyslog.go#L182-L185。RFC5424 与 RFC3164解析标准选择syslog_standard取值RFC5424默认或RFC3164。README 注释标注 RFC3164 only supports UDP transport (no streaming support)但从源码结构看createStreamDataHandler中同样存在octetcounting.NewParserRFC3164与nontransparent.NewParserRFC3164的构造分支syslog.go#L190-L205且 testcases 目录下存在rfc3164_octet_counting_strict_tcp、rfc3164_non_transparent_strict_tcp等 TCP 用例即当前实现的 TCP 流上也支持 RFC3164 解析实践中仍建议优先使用 RFC5424 以获得完整的结构化数据与时间戳信息。RFC3164 解析器还会附加rfc3164.WithYear(rfc3164.CurrentYear{})选项syslog.go#L167因为 BSD 格式的时间戳不含年份需以当前年份补全。best_effort选项指示解析器从格式不完美的消息中尽力提取有效信息关闭默认时只有格式完整的消息会被采集。该选项在流式路径通过WithBestEffort()机器选项传递syslog.go#L160-L173在数据报路径通过parser.WithBestEffort()开启syslog.go#L240-L242。Rsyslog 集成Rsyslog 可通过其远程日志转发能力将系统日志转发给 Telegraf。大多数系统的主配置分散于/etc/rsyslog.conf与/etc/rsyslog.d/目录官方文档建议把新增规则放入配置目录以简化主配置文件的后续升级。将以下内容写入/etc/rsyslog.d/50-telegraf.conf按实际目标地址调整$ActionQueueType LinkedList # use asynchronous processing $ActionQueueFileName srvrfwd # set file name, also enables disk mode $ActionResumeRetryCount -1 # infinite retries on insert failure $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down # forward over tcp with octet framing according to RFC 5425 *.* (o)127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format # uncomment to use udp according to RFC 5424 #*.* 127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format也可以改用 advanced 格式RainerScript# forward over tcp with octet framing according to RFC 5425 action(typeomfwd Protocoltcp TCP_Framingoctet-counted Target127.0.0.1 Port6514 TemplateRSYSLOG_SyslogProtocol23Format) # uncomment to use udp according to RFC 5424 #action(typeomfwd Protocoludp Target127.0.0.1 Port6514 TemplateRSYSLOG_SyslogProtocol23Format)要点双表示 TCP 加 octet 分帧与 Telegraf 端默认的framing octet-counting相配合RSYSLOG_SyslogProtocol23Format模板确保输出的是 RFC5424 消息。若需 TLS 传输请在 Telegraf 端配置tls_cert/tls_key/tls_allowed_cacerts并参考 rsyslog 官方 TLS 文档完成对端证书设置。故障排查可以用 netcat 直接向日志接收端口发送测试消息快速验证插件是否工作# TCP with octet framing echo 57 131 2018-10-01T12:00:00.0Z example.org root - - - test | nc 127.0.0.1 6514 # UDP echo 131 2018-10-01T12:00:00.0Z example.org root - - - test | nc -u 127.0.0.1 6514注意 TCP octet-counting 示例中消息前的57前缀是消息长度计数含空格这是分帧协议的一部分。源 IP 反解析source标签存储 syslog 发送端的远端 IP 地址由tags()函数在src ! 时写入见 syslog.go#L268-L297。若需将其解析为 DNS 主机名可搭配 reverse_dns 处理器 对 tag/field 中的 IP 做反向查询并生成名称字段。RFC3164 消息的识别与转换若日志中出现如下错误通常意味着对端发送的是 RFC3164 编码的消息例如某些厂商设备如 Cisco IOS 默认输出无效的 RFC3164 消息而接收端按 RFC5424 解析E! Error in plugin [inputs.syslog]: expecting a version value in the range 1-999 [col 5]一种通用做法是让 rsyslog 先接收 RFC3164 消息并转译为 RFC5424 后再转发给 Telegraf。在 rsyslog 配置中如/etc/rsyslog.d/50-telegraf.conf添加# This makes rsyslog listen on 127.0.0.1:514 to receive RFC3164 udp # messages which can them be forwarded to telegraf as RFC5424 $ModLoad imudp #loads the udp module $UDPServerAddress 127.0.0.1 $UDPServerRun 514然后将 RFC3164 消息发往端口 514由 rsyslog 完成格式转换与转发目标地址按实际部署调整。输出的指标结构每条成功解析的 syslog 消息生成一条名为syslog的指标标签与字段映射由 fields() 与 tags() 函数 实现标签tagsseverityseverity 短名如info、noticefacilityfacility 短名如daemon、local4hostname消息中的主机名appname应用名source发送端远端 IPUnix 套接字时为空字段fieldsversioninteger仅 RFC5424severity_codeintegerfacility_codeintegertimestampinteger消息内记录的时间UnixNanoprocidstring可选msgidstring可选messagestring消息体右端空白被裁剪Structured Datastring/bool指标时间戳取消息接收时间RFC3164 消息因无内嵌版本时间戳字段version字段不出现结构化数据的字段生成规则RFC5424 消息可包含多个 SD-ID、每个 SD-ID 又可带多个参数。fields()对StructuredData的处理规则syslog.go#L322-L333某 SD-ID 不带任何参数时以该 SD-ID 为名生成布尔字段true仅表示其存在带参数时每个参数生成一个字段字段名为SD_ID sdparam_separator PARAM_NAME的拼接默认分隔符_。例如输入消息170 1651 2018-10-01:14:15.000Z mymachine.example.com evntslog - ID47 [exampleSDID32473 iut3 eventSourceApplication eventID1011] An application event log entry...输出本地 line 协议格式syslog,appnameevntslog,facilitylocal4,hostnamemymachine.example.com,severitynotice exampleSDID32473_eventID1011,exampleSDID32473_eventSourceApplication,exampleSDID32473_iut3,facility_code20i,messageAn application event log entry...,msgidID47,severity_code5i,timestamp1065910455003000000i,version1i 1538421339749472344README 中给出的另一组示例输出展示了真实部署场景多个容器/主机通过 UDP 上报 daemon 类日志的形态syslog,appnamedocker-compose,facilitydaemon,hostbb8,hostnamedroplet,locationhome,severityinfo,source10.0.0.12 facility_code3i,messageredacted,severity_code6i,timestamp1624643706396113000i,version1i 1624643706400667198 syslog,appnametailscaled,facilitydaemon,hostbb8,hostnamedev,locationhome,severityinfo,source10.0.0.15 facility_code3i,messageredacted,severity_code6i,timestamp1624643706403394000i,version1i 1624643706407850408回归测试覆盖矩阵一目了然插件的解析行为由 plugins/inputs/syslog/testcases/ 下的 50 个目录化用例覆盖TestCases会遍历每个目录加载其中的telegraf.conf将input*.txt作为消息经真实 TCP/TLS/UDP/Unix 套接字发送给插件再与expected.out及可选的expected.err比对见 syslog_test.go#L172-L312。用例命名即配置组合矩阵例如octet_counting_strict_tcp_1st_avg_ok/octet_counting_best_effort_tcptlsoctet-counting × 严格/尽力解析 × TCP 与 TLS 传输non_transparent_strict_unix/non_transparent_best_effort_unixtls非透明分帧 × Unix 流套接字含 TLSrfc5424_strict_udp_min_incomplete、rfc3164_best_effort_udpRFC5424/RFC3164 × UDP 数据报含最短/最大长度边界octet_counting_maxlength_okmax_message_length限制验证rfc5424_best_effort_toolong_appname超过 RFC 长度限制的 appname 在 best effort 下的容错行为。TLS 场景使用 testutil/pki 中的证书体系在测试内完成握手。这组用例可作为理解各配置项组合效果的最佳参考。小结与适用前提生产监听推荐server tcp://ip:6514framing octet-countingsyslog_standard RFC5424均为默认值并与 Rsyslog 的(o)转发配合使用 UDP 时注意read_buffer_size与max_decompression_size对大数据报和 gzip 负载的保护作用allowed_sources可同时用于两种传输做来源限制若上游只能发 BSD 旧格式RFC3164优先让中间层 rsyslog 完成转译其次可开启best_effort并评估格式差异需要把source标签转为可读主机名时在管道中追加 reverse_dns 处理器。相关参考文件插件说明、示例配置、插件实现、通用套接字层、全局配置文档。【免费下载链接】telegrafAgent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.项目地址: https://gitcode.com/GitHub_Trending/te/telegraf创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 Telegraf Syslog 输入插件实战：从 RFC5424/RFC3164 解析到 Rsyslog 对接的完整配置指南
Telegraf Syslog 输入插件实战从 RFC5424/RFC3164 解析到 Rsyslog 对接的完整配置指南【免费下载链接】telegrafAgent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.项目地址: https://gitcode.com/GitHub_Trending/te/telegrafTelegraf 的inputs.syslog插件是一个服务类输入插件它监听通过 Unix Domain Socket、UDP、TCP 或 TLS 传输的 syslog 消息支持带或不带 octet counting 分帧将符合 RFC5424 或 BSD syslogRFC3164格式的日志消息解析为带丰富标签与字段的指标。读完本文你将掌握该插件的完整参数配置、分帧/传输协议的选择依据、与 Rsyslog 的对接方法以及底层源码中解析器与指标映射的实际实现逻辑。插件定位为什么它是 Service Inputinputs.syslog在 插件源码 中通过inputs.Add(syslog, ...)注册其Gather()方法直接返回 nil见 syslog.go#L146-L148——指标的产生不依赖采集周期而来自监听套接字上到达的数据。这带来两个与其他输入插件的关键差异全局或插件级的interval设置可能不生效--test、--test-wait、--once等 CLI 选项对该插件可能不会产生输出。从源码结构看插件在Start()中调用socket.Setup()建立监听随后按协议族分发到两条处理路径流式套接字tcp/tcp4/tcp6/unix/unixpacket走createStreamDataHandler数据报套接字udp/udp4/udp6/ip/ip4/ip6/unixgram走createDatagramDataHandler见 syslog.go#L117-L144。完整配置参考以下配置完整继承自 plugins/inputs/syslog/README.md 与 sample.conf所有选项均与源码中Syslog结构体自身字段 内嵌的 socket.Config一一对应[[inputs.syslog]] ## Protocol, address and port to host the syslog receiver. ## If no host is specified, then localhost is used. ## If no port is specified, 6514 is used (RFC5425#section-4.1). ## ex: server tcp://localhost:6514 ## server udp://:6514 ## server unix:///var/run/telegraf-syslog.sock ## When using tcp, consider using tcp4 or tcp6 to force the usage of IPv4 ## or IPV6 respectively. There are cases, where when not specified, a system ## may force an IPv4 mapped IPv6 address. server tcp://127.0.0.1:6514 ## Permission for unix sockets (only available on unix sockets) ## This setting may not be respected by some platforms. To safely restrict ## permissions it is recommended to place the socket into a previously ## created directory with the desired permissions. ## ex: socket_mode 777 # socket_mode ## Maximum number of concurrent connections (only available on stream sockets like TCP) ## Zero means unlimited. # max_connections 0 ## Read timeout (only available on stream sockets like TCP) ## Zero means unlimited. # read_timeout 0s ## Optional TLS configuration (only available on stream sockets like TCP) # tls_cert /etc/telegraf/cert.pem # tls_key /etc/telegraf/key.pem ## Enables client authentication if set. # tls_allowed_cacerts [/etc/telegraf/clientca.pem] ## Maximum socket buffer size (in bytes when no unit specified) ## For stream sockets, once the buffer fills up, the sender will start ## backing up. For datagram sockets, once the buffer fills up, metrics will ## start dropping. Defaults to the OS default. # read_buffer_size 64KiB ## Period between keep alive probes (only applies to TCP sockets) ## Zero disables keep alive probes. Defaults to the OS configuration. # keep_alive_period 5m ## Content encoding for message payloads ## Can be set to gzip for compressed payloads or identity for no encoding. # content_encoding identity ## Maximum size of decoded packet (in bytes when no unit specified) # max_decompression_size 500MB ## List of allowed source IP addresses for incoming packets/messages. ## If not specified or empty, all sources are allowed. # allowed_sources [] ## Source IP for Source-Specific Multicast (SSM / IGMPv3) # multicast_source ## Framing technique used for messages transport ## Available settings are: ## octet-counting -- see RFC5425#section-4.3.1 and RFC6587#section-3.4.1 ## non-transparent -- see RFC6587#section-3.4.2 # framing octet-counting ## The trailer to be expected in case of non-transparent framing (default LF). ## Must be one of LF, or NUL. # trailer LF ## Whether to parse in best effort mode or not (default false). ## By default best effort parsing is off. # best_effort false ## The RFC standard to use for message parsing ## By default RFC5424 is used. RFC3164 only supports UDP transport (no streaming support) ## Must be one of RFC5424, or RFC3164. # syslog_standard RFC5424 ## Character to prepend to SD-PARAMs (default _). ## A syslog message can contain multiple parameters and multiple identifiers within structured data section. ## Eg., [id1 name1val1 name2val2][id2 name1val1 nameAvalA] ## For each combination a field is created. ## Its name is created concatenating identifier, sdparam_separator, and parameter name. # sdparam_separator _ ## Maximum length allowed for a single message (in bytes when no unit specified) ## Only applies to octet-counting framing. # max_message_length 8KiB插件还支持额外的全局与插件级配置修改指标、标签、字段、创建别名、插件排序等详见 CONFIGURATION.md。地址与默认值Init 阶段的校验逻辑配置解析与校验集中在Syslog.Init()见 syslog.go#L55-L115源码确认了若干关键行为协议必填server地址必须包含://否则返回missing protocol within address错误支持的协议族流式为tcp、tcp4、tcp6、unix、unixpacket数据报为udp、udp4、udp6、ip、ip4、ip6、unixgram其他值报unknown protocol默认地址未配置server时回退为tcp://127.0.0.1:6514仅指定主机时自动补默认端口6514测试用例 TestAddressDefault / TestAddressDefaultPort 验证了这两点默认值填充framing缺省为octet-countingsyslog_standard缺省为RFC5424sdparam_separator缺省为_trailer在插件注册时初始化为LF见 syslog.go#L358-L364。这些校验行为均有对应测试TestAddressMissingProtocol、TestAddressUnknownProtocol、TestMessageSizeCustom等见 syslog_test.go。套接字层选项的底层实现socket_mode、max_connections、read_timeout、keep_alive_period、content_encoding、max_decompression_size、allowed_sources、multicast_source、read_buffer_size等选项并非Syslog自身字段而是来自内嵌的 socket.Config——该通用套接字层同时服务于多个监听类插件。源码中的几个细节值得注意数据报默认缓冲当read_buffer_size未显式设置时UDP/unixgram 监听器将读取缓冲设为64 * 1024字节datagram.go#L192-L194 及 #L241、#L252注释标明这是按 IP 报文大小取的上限来源白名单allowed_sources在数据报接收datagram.go#L79与流式连接接受stream.go#L263两处都会执行isSourceAllowed检查即该选项对流式与数据报传输同时生效读超时告警对 TCP 流式套接字设置read_timeout 0时Init()会输出Read timeout set! Connections, inactive for the set duration, will be closed!警告syslog.go#L99-L101测试TestReadTimeoutWarning断言了该日志gzip 内容编码content_encoding gzip通过internal.WithMaxDecompressionSize等选项在接收层解压datagram.go#L260-L261配合max_decompression_size限制解压后的最大尺寸防范解压炸弹。消息分帧Framing流式传输的核心概念framing选项仅对流式传输TCP/Unix 流套接字生效决定消息在字节流中如何被切分默认值为octet-counting取值含义规范出处octet-counting默认每条消息前缀数字字节长度接收端按计数读取RFC5425#section-4.3.1、RFC6587#section-3.4.1non-transparent以固定尾标trailer结束一条消息RFC6587#section-3.4.2trailer仅在framing non-transparent时有效必须取LF默认或NUL之一。在源码中非透明分帧通过nontransparent.WithTrailer(s.Trailer)传递给解析器syslog.go#L175-L177而max_message_length仅在 octet-counting 分帧下有意义——源码注释明确说明 go-syslog 在非透明分帧下将其视为 no-opsyslog.go#L182-L185。RFC5424 与 RFC3164解析标准选择syslog_standard取值RFC5424默认或RFC3164。README 注释标注 RFC3164 only supports UDP transport (no streaming support)但从源码结构看createStreamDataHandler中同样存在octetcounting.NewParserRFC3164与nontransparent.NewParserRFC3164的构造分支syslog.go#L190-L205且 testcases 目录下存在rfc3164_octet_counting_strict_tcp、rfc3164_non_transparent_strict_tcp等 TCP 用例即当前实现的 TCP 流上也支持 RFC3164 解析实践中仍建议优先使用 RFC5424 以获得完整的结构化数据与时间戳信息。RFC3164 解析器还会附加rfc3164.WithYear(rfc3164.CurrentYear{})选项syslog.go#L167因为 BSD 格式的时间戳不含年份需以当前年份补全。best_effort选项指示解析器从格式不完美的消息中尽力提取有效信息关闭默认时只有格式完整的消息会被采集。该选项在流式路径通过WithBestEffort()机器选项传递syslog.go#L160-L173在数据报路径通过parser.WithBestEffort()开启syslog.go#L240-L242。Rsyslog 集成Rsyslog 可通过其远程日志转发能力将系统日志转发给 Telegraf。大多数系统的主配置分散于/etc/rsyslog.conf与/etc/rsyslog.d/目录官方文档建议把新增规则放入配置目录以简化主配置文件的后续升级。将以下内容写入/etc/rsyslog.d/50-telegraf.conf按实际目标地址调整$ActionQueueType LinkedList # use asynchronous processing $ActionQueueFileName srvrfwd # set file name, also enables disk mode $ActionResumeRetryCount -1 # infinite retries on insert failure $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down # forward over tcp with octet framing according to RFC 5425 *.* (o)127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format # uncomment to use udp according to RFC 5424 #*.* 127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format也可以改用 advanced 格式RainerScript# forward over tcp with octet framing according to RFC 5425 action(typeomfwd Protocoltcp TCP_Framingoctet-counted Target127.0.0.1 Port6514 TemplateRSYSLOG_SyslogProtocol23Format) # uncomment to use udp according to RFC 5424 #action(typeomfwd Protocoludp Target127.0.0.1 Port6514 TemplateRSYSLOG_SyslogProtocol23Format)要点双表示 TCP 加 octet 分帧与 Telegraf 端默认的framing octet-counting相配合RSYSLOG_SyslogProtocol23Format模板确保输出的是 RFC5424 消息。若需 TLS 传输请在 Telegraf 端配置tls_cert/tls_key/tls_allowed_cacerts并参考 rsyslog 官方 TLS 文档完成对端证书设置。故障排查可以用 netcat 直接向日志接收端口发送测试消息快速验证插件是否工作# TCP with octet framing echo 57 131 2018-10-01T12:00:00.0Z example.org root - - - test | nc 127.0.0.1 6514 # UDP echo 131 2018-10-01T12:00:00.0Z example.org root - - - test | nc -u 127.0.0.1 6514注意 TCP octet-counting 示例中消息前的57前缀是消息长度计数含空格这是分帧协议的一部分。源 IP 反解析source标签存储 syslog 发送端的远端 IP 地址由tags()函数在src ! 时写入见 syslog.go#L268-L297。若需将其解析为 DNS 主机名可搭配 reverse_dns 处理器 对 tag/field 中的 IP 做反向查询并生成名称字段。RFC3164 消息的识别与转换若日志中出现如下错误通常意味着对端发送的是 RFC3164 编码的消息例如某些厂商设备如 Cisco IOS 默认输出无效的 RFC3164 消息而接收端按 RFC5424 解析E! Error in plugin [inputs.syslog]: expecting a version value in the range 1-999 [col 5]一种通用做法是让 rsyslog 先接收 RFC3164 消息并转译为 RFC5424 后再转发给 Telegraf。在 rsyslog 配置中如/etc/rsyslog.d/50-telegraf.conf添加# This makes rsyslog listen on 127.0.0.1:514 to receive RFC3164 udp # messages which can them be forwarded to telegraf as RFC5424 $ModLoad imudp #loads the udp module $UDPServerAddress 127.0.0.1 $UDPServerRun 514然后将 RFC3164 消息发往端口 514由 rsyslog 完成格式转换与转发目标地址按实际部署调整。输出的指标结构每条成功解析的 syslog 消息生成一条名为syslog的指标标签与字段映射由 fields() 与 tags() 函数 实现标签tagsseverityseverity 短名如info、noticefacilityfacility 短名如daemon、local4hostname消息中的主机名appname应用名source发送端远端 IPUnix 套接字时为空字段fieldsversioninteger仅 RFC5424severity_codeintegerfacility_codeintegertimestampinteger消息内记录的时间UnixNanoprocidstring可选msgidstring可选messagestring消息体右端空白被裁剪Structured Datastring/bool指标时间戳取消息接收时间RFC3164 消息因无内嵌版本时间戳字段version字段不出现结构化数据的字段生成规则RFC5424 消息可包含多个 SD-ID、每个 SD-ID 又可带多个参数。fields()对StructuredData的处理规则syslog.go#L322-L333某 SD-ID 不带任何参数时以该 SD-ID 为名生成布尔字段true仅表示其存在带参数时每个参数生成一个字段字段名为SD_ID sdparam_separator PARAM_NAME的拼接默认分隔符_。例如输入消息170 1651 2018-10-01:14:15.000Z mymachine.example.com evntslog - ID47 [exampleSDID32473 iut3 eventSourceApplication eventID1011] An application event log entry...输出本地 line 协议格式syslog,appnameevntslog,facilitylocal4,hostnamemymachine.example.com,severitynotice exampleSDID32473_eventID1011,exampleSDID32473_eventSourceApplication,exampleSDID32473_iut3,facility_code20i,messageAn application event log entry...,msgidID47,severity_code5i,timestamp1065910455003000000i,version1i 1538421339749472344README 中给出的另一组示例输出展示了真实部署场景多个容器/主机通过 UDP 上报 daemon 类日志的形态syslog,appnamedocker-compose,facilitydaemon,hostbb8,hostnamedroplet,locationhome,severityinfo,source10.0.0.12 facility_code3i,messageredacted,severity_code6i,timestamp1624643706396113000i,version1i 1624643706400667198 syslog,appnametailscaled,facilitydaemon,hostbb8,hostnamedev,locationhome,severityinfo,source10.0.0.15 facility_code3i,messageredacted,severity_code6i,timestamp1624643706403394000i,version1i 1624643706407850408回归测试覆盖矩阵一目了然插件的解析行为由 plugins/inputs/syslog/testcases/ 下的 50 个目录化用例覆盖TestCases会遍历每个目录加载其中的telegraf.conf将input*.txt作为消息经真实 TCP/TLS/UDP/Unix 套接字发送给插件再与expected.out及可选的expected.err比对见 syslog_test.go#L172-L312。用例命名即配置组合矩阵例如octet_counting_strict_tcp_1st_avg_ok/octet_counting_best_effort_tcptlsoctet-counting × 严格/尽力解析 × TCP 与 TLS 传输non_transparent_strict_unix/non_transparent_best_effort_unixtls非透明分帧 × Unix 流套接字含 TLSrfc5424_strict_udp_min_incomplete、rfc3164_best_effort_udpRFC5424/RFC3164 × UDP 数据报含最短/最大长度边界octet_counting_maxlength_okmax_message_length限制验证rfc5424_best_effort_toolong_appname超过 RFC 长度限制的 appname 在 best effort 下的容错行为。TLS 场景使用 testutil/pki 中的证书体系在测试内完成握手。这组用例可作为理解各配置项组合效果的最佳参考。小结与适用前提生产监听推荐server tcp://ip:6514framing octet-countingsyslog_standard RFC5424均为默认值并与 Rsyslog 的(o)转发配合使用 UDP 时注意read_buffer_size与max_decompression_size对大数据报和 gzip 负载的保护作用allowed_sources可同时用于两种传输做来源限制若上游只能发 BSD 旧格式RFC3164优先让中间层 rsyslog 完成转译其次可开启best_effort并评估格式差异需要把source标签转为可读主机名时在管道中追加 reverse_dns 处理器。相关参考文件插件说明、示例配置、插件实现、通用套接字层、全局配置文档。【免费下载链接】telegrafAgent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.项目地址: https://gitcode.com/GitHub_Trending/te/telegraf创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考