This short guide will look at the HTTP output for Logstash. HTTP is ubiquitous on the Internet. So much so that most people don’t even know they use it every day. Most API’s out there use HTTP. Logstash provides both an HTTP input and output, enabling you to connect Logstash to any API using HTTP out there. This guide focuses on the output. A guide looking at the input will follow shortly.

Quick Info

Links: Source | Documentation
Version: 1.4.2
Requirements:

  • A HTTP server to connect to.

This output gives you the ability to convert Logstash events into HTTP requests. Since most of the web operates on HTTP, this gives you the ability to integrate with the wider web without a lot of effort.

The short version

The HTTP output requires only two parameters to be configured correctly: The url to which the request should be made, and the http_method to use to make the request:

# Bare minimum
output {
  http {
    url => "http://test.eagerelk.com"
    http_method => "post"
  }
}

Logstash will now POST the Logstash events to test.eagerelk.com. The body of the request will contain the Logstash event encoded as JSON. As a basic setup this will do, but you’d probably want to customize that into something more usable. Read below for more.

The longer version

The HTTP output has enough options for you to customize it to such an extent that you can interface with almost any API. We’ll first look at a simple form submission, presumably to an old API that still relies on forms.

Simple form submission

You might need to integrate with an old system that needs it data populated using a form submission (or simple POST request). You can get the HTTP output to do that for you by simply setting the format parameter:

# Send the event as a form submission
output {
  http {
    url => "test.eagerelk.com"
    http_method => "post"
    format => "form"
  }
}

If your event contained three fields: name, surname and email, with the values John, Smith and johnsmith@gmail.com respectively, the URL endpoint will now receive a POST request with the request body equal to name=John&surname=Smith&email=johnsmith@gmail.com.

This shows only the basic options available. As a more advanced example we’ll be posting Logstash events to the Twilio API to be delivered as text messages.

Logstash to Twilio

We’ll use the mapping parameter to change the event into what the Twilio API requires. It assumes that the recipient’s number is in the to_number field. It will use the message field as the body of the text message. The format property is important in that it ensures that the content is sent as a form submission. The headers property can be used for Authorization and other headers needed to complete the request.

# Send the event to Twilio as a text message
output {
  http {
    url => "https://api.twilio.com/2010-04-01/Accounts/ASDF1234YourAccountID/Messages.json"
    http_method => "post"
    format => "form"
    mapping => [ "Body", "%{message}", "From", "+1234567890", "To", "%{to_number}" ]
    headers => [
      'Authorization', 'Basic Q1234567890AbCdEfGhIjKlMnOpQrStUvWxYz1234567890AbCdEfGhIjKlMnOpQrStUvWxYz12345678902NzNhOQ=='
    ]
  }
}

It’s a more involved setup, but it allows you to integrate with a more complex API without having to do any coding.

The format option

This option will determine the format of the message as it gets sent to the HTTP endpoint. It has three possible values: json, form and message.

JSON
In the `json` format the event will be encoded as a JSON object before it’s sent to the HTTP endpoint. It will use all the event’s fields, unless you use the `mapping` setting to specify which fields and what values should be sent.
Form
With the `form` format (which is the default) the message will be encoded as a query parameter string. Once again you can use the `mapping` setting to the fields and values to be sent.
Message
If you specify the `message` format, you also need to specify the `message` setting. You can use variable substitution to build up the value you want to send to the endpoint.

All the options

url – String – The URL the HTTP request should be sent to. There is no default, and it is required.

http_method – String – The HTTP method to use in the request. Should be one of put or post. There is no default, and it is required.

verify_ssl – Boolean – Specify whether or not the SSL certificates should be verified against the Certificate Authority. Defaults to true.

headers – Array – An array of header names followed by their values. Use this for Authentication and other customizations.

content_type – String – Use this to specify the Content Type of the request. It will default to application/json if format is json, and application/x-www-form-urlencoded if format is `form.

mapping – Array – The fields and values that should be included in the request. Use this setting to modify the content you want to send to the endpoint.

format – String – One of json, form or message. This will directly influence how the event will be formatted before it is sent to the endpoint.

message – String – If you specify the format to be message, this setting is required to specify what data should be sent to the endpoint.


Struggling with the logs? Use Logstash Config Guide to get going quickly. Try it!

Coder. Thinker. Human. I try to write good code for a living and wrangle data as a hobby. Be sure to check out the book I'm writing: The Logstash Config Guide.

Subscribe To Our Newsletter

Subscribe To Our Newsletter

Join our mailing list to receive the latest news and updates from our team.

You have Successfully Subscribed!

Share This