Logstash - Filter

Logstash menggunakan filter di tengah jalur antara input dan output. Filter tindakan Logstash memanipulasi dan membuat acara sepertiApache-Access. Banyak plugin filter yang digunakan untuk mengelola acara di Logstash. Di sini, dalam contoh fileLogstash Aggregate Filter, kami memfilter durasi setiap transaksi SQL dalam database dan menghitung total waktu.

Memasang Plugin Filter Agregat

Menginstal Plugin Filter Agregat menggunakan utilitas Logstash-plugin. Plugin Logstash adalah file batch untuk windowsbin folder di Logstash.

>logstash-plugin install logstash-filter-aggregate

logstash.conf

Dalam konfigurasi ini, Anda dapat melihat tiga pernyataan 'jika' untuk Initializing, Incrementing, dan generating total durasi transaksi, yaitu sql_duration. Plugin agregat digunakan untuk menambahkan sql_duration, yang ada di setiap peristiwa log input.

input {
   file {
      path => "C:/tpwork/logstash/bin/log/input.log"
   }
} 
filter {
   grok {
      match => [
         "message", "%{LOGLEVEL:loglevel} - 
            %{NOTSPACE:taskid} - %{NOTSPACE:logger} - 
            %{WORD:label}( - %{INT:duration:int})?" 
      ]
   }
   if [logger] == "TRANSACTION_START" {
      aggregate {
         task_id => "%{taskid}"
         code => "map['sql_duration'] = 0"
         map_action => "create"
      }
   }
   if [logger] == "SQL" {
      aggregate {
         task_id => "%{taskid}"
         code => "map['sql_duration'] ||= 0 ;
            map['sql_duration'] += event.get('duration')"
      }
   }
   if [logger] == "TRANSACTION_END" {
      aggregate {
         task_id => "%{taskid}"
         code => "event.set('sql_duration', map['sql_duration'])"
         end_of_task => true
         timeout => 120
      }
   }
}
output {
   file {
      path => "C:/tpwork/logstash/bin/log/output.log"    
   }
}

Jalankan Logstash

Kita bisa menjalankan Logstash dengan menggunakan perintah berikut.

>logstash –f logstash.conf

input.log

Blok kode berikut menunjukkan data log masukan.

INFO - 48566 - TRANSACTION_START - start
INFO - 48566 - SQL - transaction1 - 320
INFO - 48566 - SQL - transaction1 - 200
INFO - 48566 - TRANSACTION_END - end

output.log

Seperti yang ditentukan dalam file konfigurasi, pernyataan 'jika' terakhir di mana pencatat berada - TRANSACTION_END, yang mencetak total waktu transaksi atau durasi_sql. Ini telah disorot dengan warna kuning di output.log.

{
   "path":"C:/tpwork/logstash/bin/log/input.log","@timestamp": "2016-12-22T19:04:37.214Z",
   "loglevel":"INFO","logger":"TRANSACTION_START","@version": "1","host":"wcnlab-PC",
   "message":"8566 - TRANSACTION_START - start\r","tags":[]
}
{
   "duration":320,"path":"C:/tpwork/logstash/bin/log/input.log",
   "@timestamp":"2016-12-22T19:04:38.366Z","loglevel":"INFO","logger":"SQL",
   "@version":"1","host":"wcnlab-PC","label":"transaction1",
   "message":" INFO - 48566 - SQL - transaction1 - 320\r","taskid":"48566","tags":[]
}
{
   "duration":200,"path":"C:/tpwork/logstash/bin/log/input.log",
   "@timestamp":"2016-12-22T19:04:38.373Z","loglevel":"INFO","logger":"SQL",
   "@version":"1","host":"wcnlab-PC","label":"transaction1",
   "message":" INFO - 48566 - SQL - transaction1 - 200\r","taskid":"48566","tags":[]
}
{
   "sql_duration":520,"path":"C:/tpwork/logstash/bin/log/input.log",
   "@timestamp":"2016-12-22T19:04:38.380Z","loglevel":"INFO","logger":"TRANSACTION_END",
   "@version":"1","host":"wcnlab-PC","label":"end",
   "message":" INFO - 48566 - TRANSACTION_END - end\r","taskid":"48566","tags":[]
}