Grand Challenge: High Performance Stream Queries in Scala
Traffic monitoring is an important stream processing application which is highly dynamic and requires aggregation of
spatially colocated data. Inspired by this, the DEBS 2015
Grand Challenge uses publicly available taxi transportation
information to compute online the most frequent routes and
most profitable areas. In this paper, we describe our solution
to the DEBS 2015 Grand Challenge written in Scala. Our
large-scale solution employs Apache Spark, while our challenge implementation is highly specialized and can process
events at a 10 ms latency and at a throughput of 114,000
events per second.
Tags: grand challenge, spark, scala, taxi monitoring
Readers who enjoyed the above work, may also like the following:
- Solving Manufacturing Equipment Monitoring Through Efficient Complex Event Processing.
Tilmann Rabl, Kaiwen Zhang, Mohammad Sadoghi, Navneet Kumar Pandey, Aakash Nigam, Chen Wang, and Hans-Arno Jacobsen.
In Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, 2012.
Best DEBS Challenge Award - Public Voting.
Tags: event processing, grand challenge
- DL-Store: A Distributed Hybrid OLTP and OLAP Data Processing Engine.
Kaiwen Zhang, Mohammad Sadoghi, and Hans-Arno Jacobsen.
In ICDCS Demos, 2016.
Tags: olap/oltp, data store, distributed transactions, spark