ArangoDB, a provider of native multi-model NoSQL database solutions, announced the latest findings of its open source NoSQL performance benchmark series. To enable vendors to respond to the results and contribute improvements, ArangoDB has published the necessary scripts required to repeat the benchmark.

The goal of the benchmark is to measure the performance of each database system when there is no cache used. The benchmark is completely open source and therefore driven by community input.

For the 2018 benchmark, three leading single-model database systems were compared against ArangoDB: Neo4j for graph; MongoDB for document; and PostgreSQL for relational database. Additionally, it tested ArangoDB against a multi-model database, OrientDB. The latest GA versions (as of January 26, 2018) of database systems were used: Neo4j 3.3.1, MongoDB 3.6.1, PostgreSQL 10.1 (tabular & jsonb), OrientDB 2.2.29 and ArangoDB 3.3.3.

A simple client/server set up was used, alongside instances AWS recommends for both relational and non-relational databases. The benchmark used NodeJS 8.9.4. The operating system for the servers was Ubuntu 16.04, including the OS-patch 4.4.0-1049-aws — this includes Meltdown and Spectre V1 patches. Each database had an individual warm-up.

The benchmark results below demonstrate how a multi-model database competes and often outperforms single-model databases in their speciality. In fundamental queries like Single Read, Single Write and Single Write Sync, ArangoDB achieved positive results and outperformed PostgreSQL. Note, lower percentages indicate higher throughput, higher percentages indicate lower throughput.

To test aggregation, the benchmark analysed age distribution in a social network. It aggregated over a single collection (1,632,803 documents). It computed statistics about age distribution in the network by counting which age occurs most frequently. It did not put a secondary index for this attribute on any of the databases, so they all had to perform a full collection scan and do counting statistics – a typical ad-hoc query to perform.

Computing the aggregation in ArangoDB took on average 1.07sec. As expected, PostgreSQL, as the representative of the relational world, demonstrated the best performance with 0.3sec (but only when the data was stored as tabular). For the same task, but with data stored as a JSONB document, PostgreSQL needed more time when compared to MongoDB and more than twice the time when compared to ArangoDB. Since the last benchmark, OrientDB hasn’t improved and is slower by 20x.

Further benchmark results include neighbors search and neighbors with profile data, shortest path and memory usage. The full open source NoSQL performance benchmark is available here with full details about data, machines, software and tests.

Claudius Weinberger, CEO of ArangoDB, said: “One of our main objectives, when conducting the benchmark, is to demonstrate that a native multi-model database can compete with single-model databases on their home turf. Our findings illustrate again that native multi-model ArangoDB not only competes, but frequently outperforms single-model.

The feedback we receive from this benchmark enables us to improve by listening and reacting to community opinion. To get more developers to buy-in to the multi-model approach, ArangoDB needs to continually evolve and innovate.”

Comment on this article below or via Twitter: @ VanillaPlus OR @jcvplus