Can You Pass the ACID Test?

If you grew up in the ’60s then an ACID Test was a phrase associated with Ken Kesey (author of One Flew Over the Cuckoo’s Nest) as part of his investigations into hallucinogenic drugs. However, the meaning of an acid test goes back much further – to the gold rush – when acid was used by miners to determine if they had found the riches they were seeking. Since then, it has come to have a more general meaning, neatly summarized as: A sure test, giving an incontestable result.

In the world of MarkLogic and enterprise databases in general, ACID is another acronym. It does not refer to a test at all but the allusion is relevant since, simply put, MarkLogicians would say that ACID compliance means “that we don’t lose your data.”

This seemed like a somewhat limited explanation for such an important differentiator of our Enterprise NOSQL offering. I wanted to understand what the emphasis on ACID compliance really meant and be able to explain it simply. So here comes my Light Summary Description (LSD) for ACID:

ACID is an acronym which stands for four properties of a database which mean that it processes transactions reliably:

• A is for Atomicity: The database must execute the whole transaction and not just part of it – all or nothing.
• C is for Consistency: The database must enforce the system rules, e.g. you can’t put a value in a field if that value is not allowed.
• I is for Isolation: If multiple transactions are running at the same time they will run independently and the end result will be as if they were executed one after the other.
• D is for Durability: Once a transaction is done, it stays done (even if the system crashes thereafter).

In most NOSQL databases these properties are relaxed or are non-existent in favour of performance. This may be acceptable in a prototype or non-mission critical application but it is hard to imagine why most organisations would want to work with their vital data assets in this way. In addition many databases rely on locking to guarantee ACID compliance. Locking is the process of not allowing data that is being used in one transaction to be changed by another transaction until the first one is finished. Locking can cause serious performance issues for some users if others are running complex, lengthy queries. It is not an issue in MarkLogic thanks to multi-version concurrency control and time-stamping. You can learn more about locking and many other MarkLogic features from Jason Hunter, Chief Architect at MarkLogic, in his updated White Paper Inside MarkLogic Server.

Just how important is ACID compliance? Jason writes: “ACID transactions are considered commonplace for relational databases but they’re a game changer for document-centric databases and search-style queries.” Philip Howard of Bloor Research emphasized the importance of ACID compliance as one of MarkLogic’s differentiators in his recent article on NOSQL databases: “they have over 350 real, live, actual paying enterprise customers (many of them major companies) and, perhaps most importantly, the product is ACID compliant. That seems like a contradiction in terms doesn’t it? NoSQL and ACID compliant!”

In addition, the cost of not having ACID compliance is something that those 350+ MarkLogic customers decided was too high. Take durability as an example – they decided that it was NOT acceptable to lose completed transactions in the event of a complete system failure. They wanted to bring the system back up from disaster knowing that everything that had run up to that point was still there, finished and waiting to be used – no doubt about it.

Thus you can be sure that as a result of ACID compliance, a transaction executed in a MarkLogic database will deliver “an incontestable result”. So next time you are talking to one of the “plethora of suppliers that most of us have never heard of” in the NOSQL space, perhaps the first thing you should ask them is: Can you pass the ACID test?