Baidu swaps DRAM for Optane to power in-memory database

Although slower than DRAM, Baidu says the swap will lower its costs.

Off the back of a three-year collaboration agreement signed earlier this month between Intel and Baidu, the pair have announced that Baidu is adding Optane DC persistent memory to its Feed Stream service.

Baidu’s aggregation and recommendation engine, Feed Stream, relies on an in-memory database called Feed-Cube, which will run on systems using both traditional DRAM and Intel’s Optane memory, before being migrated to systems solely using Optane.

Driving the Chinese giant to look at DRAM replacements was overall cost, and after testing NVMe SSDs, which were unsurprisingly slower than DRAM, Baidu settled on using Optane to store Feed-Cube’s data files while DRAM retained the hash tables.

In a test of 200,000 queries per second fetching 100 key-value pairs, the DRAM+ Optane system was 24% slower on access times and used 7% more CPU than the DRAM-only system.

At the same time, single-server DRAM usage drops by more than half, which will undoubtedly reduce costs in terms of the petabyte-level storage capacity of Feed-Cube,” the pair said.

Moving to a full Optane system required Baidu to create a memory allocation library for Optane, to replace the standard malloc and free calls programmers are used to.

Testing the three configurations against 500,000 queries per second found pure Optane system was just under 10% slower than pure DRAM, with the hybrid system sitting between them.

Baidu said it was looking at using Optane elsewhere in its business, such as in cases of power failure where services need to be reloaded from drives and can take up to 10 minutes to be restored.

With the high-speed read-and-write performance and non-volatility of Intel Optane DC persistent memory, the loading time is now reduced to just seconds,” it said.

The company is also looking to add Optane into its systems running Apache Spark.

In April, Intel announced a new member of its Optane series, the H10, which adds some of the company’s 3D NAND storage beside its Optane 3D XPoint memory on a single M.2 drive.

Intel unveiled its Optane DC persistent memory in May 2018.

For its second quarter results posted earlier this month, Baidu recorded steady revenue of 26.3 billion yuan ($3.84 billion) for the three months to June 2019, while net income dropped 64% to 2.41 billion yuan.

Source: https://www.zdnet.com/

Related presentations

Liked this post?

Share it on your social networks