The Project Layout

Here is what you can find in each subproject in the S2Graph source tree.

1. s2core: The core library, containing the data abstractions for graph entities, storage adapters and utilities.
2. s2rest_play: The REST server built with Play framework, providing the write and query API.
3. s2rest_netty: The REST server built directly using Netty, implementing only the query API.
4. loader: A collection of Spark jobs for bulk loading streaming data into S2Graph.
5. spark: Spark utilities for loader and s2counter_loader.
6. s2counter_core: The core library providing data structures and logics for s2counter_loader.
7. s2counter_loader: Spark streaming jobs that consume Kafka WAL logs and calculate various top-K results on-the-fly.

The first three projects are for OLTP-style workloads, currently the main target of S2Graph. The other four projects could be helpful for OLAP-style or streaming workloads, especially for integrating S2Graph with Apache Spark and/or Kafka. Note that, the latter four projects are currently out-of-date, which we are planning to update and provide documentations in the upcoming releases.