Implementing your first custom pipeline
Once you're up and running, you may want to start writing custom extractors and transformers. The easiest way to get started is to download the streamliner-starter repository from GitHub. It contains examples of extractors and transformers pre-written and configured to compile out of the box.
For more examples of extractors and transformers, download the streamliner-examples repository from GitHub. It contains several relatively simple examples of extractors and transformers, and a more complex example that processes Thrift objects.
For advanced users
For advanced users, the memsql-spark-connector repository contains the code we use to read and write data to and from MemSQL. It also contains the code for the MemSQL Spark Interface, which is the application we use to run and control ETL pipelines on Spark. Users interested in the low-level details of how we connect to MemSQL and to Spark may be interested in the contents of this repo, but it's not necessary for running your own pipelines.