Since both Slack and Logstash have IRC integrations, I decided to try and connect Elasticsearch to a Slack instance. Here’s how to do it:
Slack Setup
Access to slack using IRC is set up per user. So you can either use your current user, or create a dedicated user for the integration. It’s probably better to create a new user, since you’ll be storing the login credentials in plain text wherever you’re running Logstash.
Once you’ve decided on a user, log in with that user, and go to your “Account & Profile” settings. A little way down on the main “Settings” tab you’ll find a “Gateways” section. Click through to that. If your team’s gateway settings haven’t been set up properly yet, you’ll find a link to the admin settings as well. Ensure that the IRC gateway is enabled.
If every thing has been set up correctly, you’ll find the IRC (and XMPP) settings you need to connect here. Record those somewhere, and proceed to the Logstash setup.
Logstash Setup
The first thing we’ll set up in Logstash is the IRC input. Use the values from the Slack IRC gateway here:
input {
irc {
# The host and channels settings are required by Logstash
host => "myteam.irc.slack.com"
channels => [ "#general", "#random" ]
# The following settings are required by Slack
nick => "logstash_user"
user => "logstash_user"
password => "myteam.12345AbCd4321DcBa"
real => "Logstash Integration"
secure => true
# It's a good idea to add the type for Elasticsearch
type => "slack"
}
}
That takes care of pulling messages from Slack. The config above only tracks the general and random rooms, but you can add any rooms you want. I tried connecting to individual users using the ‘@’ instead of ‘#’, but that didn’t seem to work. Logstash will now get retrieve messages in the configured rooms, and send them through it’s event pipeline. You can now set up filters (if needed) and an output.
Elasticsearch
A vanilla install of Elasticsearch is enough for our needs. Logstash (by default) manages mappings and indices, so unless you’ve explicitly disabled it in Logstash, you can just fire up Elasticsearch and get going. The first order of business is to configure the Elasticsearch output in Logstash:
output {
elasticsearch {
protocol => "http"
}
}
You will need to set the host and port settings if Elasticsearch is listening on something other than the defaults. If you’re having trouble connecting to Elasticsearch using the defaults, try setting protocol to http and use port 9200. It’s not as efficient as the default node connection, but it’s much easier to set up.
Simple enough. Logstash will now push any messages posted in the configured rooms to Elasticsearch. From there you can use any Elasticsearch client or dashboard to analyze and search your Slack messages.
Jurgens du Toit
Coder. Thinker. Human. I try to write good code for a living and wrangle data as a hobby. Be sure to check out the book I'm writing: The Logstash Config Guide.
Subscribe To Our Newsletter
Join our mailing list to receive the latest news and updates from our team.
