Order the mess : ACID ,CAP theorem and NoSQL

Database transaction
Lets define a transaction as unit  of work that is reliable and recoverable. By definition transaction must support ACID features :
 Atomic - if transaction contains several actions it can occur in all-or-nothing manner.
Consistent - if i commit some action - all other actions see the change immediately.
Isolated - 2 transactions that run on the same data will see different copies until on of them commits changes.
Example
1 transactions reads some value when other updates it.Untill transaction B commits - transaction A will see an old value.
Durable -Once transaction is committed -even if the system crushes, the transaction results not going to get lost.Its achieved by managing transaction logs that can be replayed,

Great! we have 4 properties of a transaction.

All those properties achieved in different ways .


CAP theorem
Its set of  requirements for distributed system
Consistency- all servers in the distributed system will be in the same state.
Availability - you will be always be able to get data out of the system ( even if isn't consistent)
Partition tolerance - service will stay available even if some parts of he system crushes.

So its proved that all 3 of them can't be achieved at the same time.And its required to have 2 out of 3.

So lets make some order :
1. CAP are properties of distributed system.
2. ACID are futures of transaction .

Transactions are futures of relational databases. like MySQL, Oracle etc.

Big Data era.

Amounts of data  that is being generated in last years caused  scale and performance to be most important over some ACID futures. Providing Consistency for example causes the system to be much slower. And sometimes we can live with an idea that the data will not be consistency at the same second ( but finally yes!)

That causes the idea of other DB systems to get popular . Some of them provide parts of ACID , some of them don't have SQL as main interface. And we call them NoSQL databases.
Ideas of Relational storage is avoided in some of them.
But as main concept they provide 2 out of 3 CAP requirements, avoid ACID and solve some predefined problem in data storage, like graph relations, columnar storage etc.

Which means the main focus now is on data instead of the system.You look, investigate the data then you look for storage solution instead of taking some Relational storage and starting modelling the data .




http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf
http://en.wikipedia.org/wiki/Eventual_consistency
http://www.cs.berkeley.edu/~istoica/classes/cs268/06/notes/20-BFTx2.pdf

NoSQL - types and use cases

In order to explain what NoSQL is and why its needed lets first introduce what was before it came into the big game...and before we knew RDBMS what provided us the main concept - ACID.
Atomicity- transactivity of actions - if we have a transaction that contains actions A,B and C - all those actions should success .If one of them fails - we shoudl rollback the previous one's to initial state.
Consistency - if tranascation A is ececuted and it should do 2 actions : 1. incease balance of account a1 in 200$  and action 2. should decrease account b1 in 100$ onec transaction is done (in success!) both numbers will be updated ( we dont care what happens during transaction execution. consistency is about what happens in the end)
Isolation - transactions that executed in parallel dont impact each other.
Durability - we dont care if the electiricty goes down in the whole area or the machine totally went down - once transaction was done - even if the system goes down - when it gets up again - the transaction result will be updated.

Great!the concept is clear and here main keyplayer come...
ORACLE , SQLServer, MySQL,DB2 etc..
Everything went fine..till the data started growing with huge speed.... then it started beeing clear that setting up 1 single machine is not enough anymore.In addition to that oracle licensing is based on CPU amount on the machine...making simple map shows us that something totally different should come and replace RDBMS....not yet! Let's shot a last bullet ..last nail to the grave of RDBMS...
CAP theorem(Brewer's theorem)
It says that its impossible in destributed computing provide Consistency,Isolation and Partition tolerance.
So lets summarize:
We have ACID (Oracle) and we want to add more machines to reach scalability.Then we go into destributed computing.And then comes CAP theorem and kicks our a**.
So what know?
We want ACID,we buy license but we cant buy it?
So one option is  to use open source RDBMS and save money...
But what if i say that you you are still in trouble? Your system based on RDBMS cant be infinitly scalible..
Once you reach to petabyte - you will be soooooo slow that you buisness will crush?
Then NoSQL comes to the game...
And what you have there?
*Open Source
*Greatly scalible
*Not Relational
*Destributed
system that during last 10 years proved itself in different companies.
We have 4 families:

• Column: Hbase, Accumulo
• Document: MongoDB, Couchbase
• Key-value : Dynamo, Riak, Redis, Cache, Project Voldemort
• Graph: Neo4J, Allegro, Virtuoso

And each family has its advantages and disadvantages:

Data Model	Performance	Scalability	Flexibility	Complexity	Functionality
Key-Value Stores	high	high	high	none	variable (none)
Column Store	high	high	moderate	low	minimal
Document Store	high	variable (high)	high	low	variable (low)
Graph Database	variable	variable	high	high	graph theory
Relational Database	variable	variable	low	moderate	relational algebra.

So you have different databases and the you defenetly say : so which one to pick?
Thats what i've asked first time i saw it. Sometimes i needed graph and sometimes relational and sometimes key-value...
So the answer is simple : you can hold all of them... and combine them...

Sources;
http://en.wikipedia.org/wiki/NoSQL
http://db-engines.com/en/ranking
http://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-techniques/


PS: if you look into db-engines link you can see top DB engines rank.Sure!Oracle Number 1. But im almost sure that NoSQL and other technologies will make with oracle the same Linux did to Windows.Sure!ORacle will stay forever...but world around it will change..