fluentd

環境 [#c0e2e919]

-CentOS6.7 -nginx 1.0.15 -mongod 2.6.11 -td-agent 2.3.0

fluentd側設定 [#lc8ef0aa]

nginxの場合 [#o650eb81]

 <source>
 type tail
 path /var/log/nginx/access.log
 pos_file /var/log/td-agent/access.pos
 tag nginx.access
 #format nginx
 format /^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)" "(?<forwarder>[^\"]*)")?/
 time_format %d/%b/%Y:%H:%M:%S %z
 </source>
 <match nginx.access>
 type copy
 
 <store>
  type mongo
  host localhost
  port 27017
  database fluentd
  collection test
  capped
  capped_size 1024m
  flush_interval 10s
 </store>
 </match>
 ※mongodbのデフォルト待受ポートが27017
 ※特にDBなどは作成しなくてもOK

確認手順 [#qa6ac30b]

 # mongo
 > show dbs
 admin    (empty)
 fluentd  2.077GB
 local    0.078GB
 test     (empty)
 > use fluentd
 switched to db fluentd
 > show collections
 system.indexes
 test
 > db.test.find()
 { "_id" : ObjectId("56b98275e138231169000007"), "remote" : "192.168.33.1", "host" : "-", "user" : "-", "method" : "GET", "path" : "/jopdsfa", "code" : "404", "size" : "571", "referer" : "-", "agent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36", "forwarder" : "-", "time" : ISODate("2016-02-09T06:08:53Z") }