Well well well, this one came a long way. I did some serious Yak shaving last week with moving the project I’m working on from Logstash-shipper (formerly Lumberjack) to Filebeat (the somehow new kid on the block).

After enabling Filebeat I started to see a lot of errors on Logstash itself (I had the output set to debug to make sure I saw all errors casted into stdout).

{:timestamp=>”2015-11-26T14:26:06.417000-0800″, :message=>”Beats input: unhandled exception”, :exception=>#<TypeError: The field ‘@timestamp’ must be a (LogStash::Timestamp, not a String (2015-11-26T22:24:33.640Z)>, :backtrace=>[“/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.0.0-java/lib/logstash/event.rb:138:in `[]=’”, “/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-0.9.2/lib/logstash/inputs/beats.rb:138:in `create_event’”, “org/jruby/RubyHash.java:1342:in `each’”,

[…]

, :level=>:error}

A thing that caught my eye at some point was the version number 0.9.2 of the logstash-input-beats plugin for Logstash.

After some searching on the Interwebs I found the following thread within the elastic forums leading to the following solution.

Solution for the problem is to MANUALLY (yes that thing that you do with your bare hands) change the beats input plugin from 0.9.2 to 0.9.3

# download 0.9.3 from rubygems
$ curl -O /tmp/logstash-input-beats-0.9.3gem https://rubygems.org/downloads/logstash-input-beats-0.9.3.gem

# uninstall current plugin
$ bin/plugin uninstall logstash-input-beats

# install 0.9.3
$ bin/plugin install /tmp/logstash-input-beats-0.9.3gem

After that you should be able to restart Logstash and everything should work as expected.