{"id":2537,"date":"2016-01-25T13:27:18","date_gmt":"2016-01-25T04:27:18","guid":{"rendered":"http:\/\/blog.rutake.com\/techmemo\/?p=2537"},"modified":"2016-01-25T13:27:18","modified_gmt":"2016-01-25T04:27:18","slug":"fluentdtd-agent-%e3%83%95%e3%82%a1%e3%82%a4%e3%83%ab%e3%81%a8s3%e3%81%ab%e3%83%ad%e3%82%b0%e3%82%92%e9%80%81%e4%bf%a1","status":"publish","type":"post","link":"https:\/\/blog.rutake.com\/techmemo\/2016\/01\/25\/fluentdtd-agent-%e3%83%95%e3%82%a1%e3%82%a4%e3%83%ab%e3%81%a8s3%e3%81%ab%e3%83%ad%e3%82%b0%e3%82%92%e9%80%81%e4%bf%a1\/","title":{"rendered":"Fluentd(td-agent) \u30d5\u30a1\u30a4\u30eb\u3068S3\u306b\u30ed\u30b0\u3092\u9001\u4fe1"},"content":{"rendered":"

match\u3067\u8907\u6570\u306e\u30a2\u30a6\u30c8\u30d7\u30c3\u30c8\u30bd\u30fc\u30b9\u306b\u4fdd\u5b58\u3057\u305f\u3044\u6642\u304c\u3042\u308b\u3002<\/p>\n

\u305d\u306e\u5834\u5408\u306fcopy\u30d7\u30e9\u30b0\u30a4\u30f3\u3092\u5229\u7528\u3059\u308b\u3068\u8907\u6570\u306e\u30a2\u30a6\u30c8\u30d7\u30c3\u30c8\u3092\u6307\u5b9a\u3067\u304d\u308b\u3002
\n\u4eca\u56de\u306e\u4f8b\u3067\u306f\u30d5\u30a1\u30a4\u30eb\u306b\u66f8\u304d\u51fa\u3057\u3064\u3064\u3001\uff11\u6642\u9593\u3054\u3068\u306bS3\u306bApache\u30ed\u30b0\u30d5\u30a1\u30a4\u30eb\u3092\u9001\u4fe1\u3057\u3066\u3044\u308b\u3002
\ntime_slice_format %Y%m%d_%H_apache.log\u304c\u305d\u306e\u8a2d\u5b9a\u3067\u3042\u308a\u3001\u30b5\u30a4\u30ba\u304c\u3042\u3075\u308c\u306a\u3044\u9650\u308a\u3001\u3053\u306e\u30d5\u30a1\u30a4\u30eb\u540d\u5358\u4f4d\u3067S3\u306b\u9001\u4fe1\u3055\u308c\u308b\u3002\u4e0b\u8a18\u4f8b\u3067\u306f\u3001\u5e74\u6708\u65e5_\u6642\u9593_apache.log\u3068\u306a\u3063\u3066\u3044\u3066\u3001\u4e00\u6642\u9593\u5358\u4f4d\u3067\u30d5\u30a1\u30a4\u30eb\u3092\u307e\u3068\u3081\u3066S3\u306b\u9001\u4fe1\u3059\u308b\u3002time_slace_wait\u304c\u305d\u306e\u6642\u9593\u304c\u7d42\u308f\u3063\u305f\u3068\u304d\u306b\u4f55\u5206\u5f85\u3064\u304b\u3068\u3044\u3046\u8a2d\u5b9a\u5024\u3060\u304c\u3001\u30c7\u30d5\u30a9\u30eb\u30c8\u306f\uff11\uff10\u5206\u306a\u306e\u3067\u6bce\u6642\uff11\uff10\u5206\u904e\u304e\u306bS3\u306b\u9001\u4fe1\u3055\u308c\u308b\u3002
\nS3\u306b\u9001\u4fe1\u3055\u308c\u308b\u5185\u5bb9\u306ffluentd\u306ejson\u5f62\u5f0f\u306e\u30d5\u30a1\u30a4\u30eb\u306e\u3088\u3046\u3060\u3002<\/p>\n

[shell]
\n# Apache logtype tail
\n format apache
\n path \/var\/log\/httpd\/blog_access_log
\n tag apache.access
\n pos_file \/tmp\/fluentd-apache.pos
\n<\/source>
\n
\n type copy
\n
\n type file
\n path \/var\/tmp\/apache.log
\n <\/store>
\n
\n type s3
\n buffer_path \/var\/tmp\/fluentd_s3
\n buffer_chunk_limit 100m #1chunk\u306b\u4fdd\u5b58\u3067\u304d\u308b\u30c7\u30fc\u30bf\u30b5\u30a4\u30ba\u4e0a\u9650
\n buffer_queue_limit 3 #1queue\u306b\u4fdd\u5b58\u3067\u304d\u308bchunk\u6570\u306e\u4e0a\u9650
\n retry_wait 30s #\u518d\u9001\u3092\u5b9f\u884c\u3059\u308b\u307e\u3067\u306e\u5f85\u3061\u6642\u9593
\n retry_limit 5 #\u518d\u9001\u5b9f\u65bd\u56de\u6570
\n aws_key_id “PUT YOUR AWS_KEY_ID”
\n aws_sec_key “PUT YOUR AWS KEY”
\n s3_region ap-northeast-1
\n s3_bucket \u30d0\u30b1\u30c3\u30c8\u540d
\n path logs\/
\n #\u6642\u9593\u5358\u4f4d\u3067\u30ed\u30b0\u30d5\u30a1\u30a4\u30eb\u3092\u751f\u6210
\n time_slice_format %Y%m%d_%H_apache.log
\n # \u30c7\u30d5\u30a9\u30eb\u30c8\u306f10\u5206\u5f85\u3063\u3066\u304b\u3089\u30a2\u30c3\u30d7\u3059\u308b\u3002
\n# time_slice_wait 1m
\n <\/store>
\n<\/match>
\n[\/shell]<\/p>\n","protected":false},"excerpt":{"rendered":"

match\u3067\u8907\u6570\u306e\u30a2\u30a6\u30c8\u30d7\u30c3\u30c8\u30bd\u30fc\u30b9\u306b\u4fdd\u5b58\u3057\u305f\u3044\u6642\u304c\u3042\u308b\u3002 \u305d\u306e\u5834\u5408\u306fcopy\u30d7 … \u7d9a\u304d\u3092\u8aad\u3080 →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[458],"tags":[437,449],"_links":{"self":[{"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/posts\/2537"}],"collection":[{"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/comments?post=2537"}],"version-history":[{"count":1,"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/posts\/2537\/revisions"}],"predecessor-version":[{"id":2538,"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/posts\/2537\/revisions\/2538"}],"wp:attachment":[{"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/media?parent=2537"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/categories?post=2537"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.rutake.com\/techmemo\/wp-json\/wp\/v2\/tags?post=2537"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}