Middleware Systems Research Group

University of Toronto, Canada

• << Publications

Introducing the Big Data Benchmarking Community

Abstract

The Workshop on Big Data Benchmarking (WBDB2012), held on May 8-9, 2012 in San Jose, CA, marked the first of a series of workshops aimed at developing industry-standard Big Data benchmarks. The workshop was attended by 60 invitees representing 45 different organizations both from industry and academia. Attendees were chosen based on their experience and expertise in one or more areas of Big Data, database systems, performance benchmarking, and big data applications. They agreed that there was both a pressing need and an opportunity for defining benchmarks to capture the end-to-end aspects of big data applications. In presentations and working sessions the workshop participants laid the foundation for future workshops by agreeing on key aspects of future Big Data benchmarks, e.g. the need to include metrics for performance as well as price/performance; the need to consider several costs, including total system cost, setup cost, and energy costs; and the need for an end-to-end benchmark serving the purposes of competitive marketing as well as product improvement. As a result of this meeting an informal “Big Data benchmarking community” has been formed, hosted by the San Diego Supercomputer Center, UC San Diego. Biweekly phone conferences are being used to keep this group engaged and to share information among members. Several benchmarking activities were started by members of this community. These range from simple examples of end-to-end benchmarks to complex components setups. Within our lightning talk and poster, we will introduce the Big Data Benchmarking Community and give an overview of current developments in Big Data benchmarking. The next Big Data Benchmarking workshops are currently planned for December 17-18, 2012 in Pune, India and June 2013 in Xian, China.

Download

Tags: big data, benchmarking