Introducing Streamliner

The simplest way to manage real-time data pipelines

Getting started

To get started with Streamliner, first download and install MemSQL. MemSQL Ops comes bundled with everything you need to create your first pipeline.

Implementing your first custom pipeline

Once you're up and running, you may want to start writing custom extractors and transformers. The easiest way to get started is to download the streamliner-starter repository from GitHub. It contains examples of extractors and transformers pre-written and configured to compile out of the box.

Next steps

For more examples of extractors and transformers, download the streamliner-examples repository from GitHub. It contains several relatively simple examples of extractors and transformers, and a more complex example that processes Thrift objects.

For advanced users

For advanced users, the memsql-spark-connector repository contains the code we use to read and write data to and from MemSQL. It also contains the code for the MemSQL Spark Interface, which is the application we use to run and control ETL pipelines on Spark. Users interested in the low-level details of how we connect to MemSQL and to Spark may be interested in the contents of this repo, but it's not necessary for running your own pipelines.