If your data lacks integrity you are, in a word, doomed.

Recently I ran across across a website and their related blog ( http://channelmeter.com/ & http://insidechannelmeter.wordpress.com/ ) . The company's focus is the analytical data of You Tube. Naturally with the size of You Tube the analytical data is going to be big.  So I knew this would not be a company that is just taking a few hundred names from a web form each day. I saw from  their blog posting that they use MySQL community edition 5.5. I reached out to asked them a few questions. “We are storing tens of millions of rows of data, and adding millions more every day.” – Dave Storrs Co-Founder, ChannelMeter.com. To me this is showing, once again, how well a company can be built and run with an open source database and tools.

 

They rely on the default InnoDB storage engine in MySQL because of the ACID compliance and data integrity it offered them. This being supported by the use of transactions and foreign keys. “If your data lacks integrity you are, in a word, doomed.”– Dave Storrs. The tradeoff, if any, for this stability was speed. They can still address some speed issues with faster machines, indexes and additional use of caching. They currently have plans to utilize a 10 minute caching timeout and deploy Varnish. 

 

As with any start up, they have run across a few issues. The rapid growth of their data has required them to evaluate their hard drives and options of partitioning or sharding their data. They have used the MySQL community to help them with any problems. “Whenever I encounter difficulties related to the DB, the answer is almost always to be found in the documentation or the forums.” – Dave Storrs. 

 

While I might be biased, I am still shocked that is it not standard to run a web server and company on a LAMP stack environment. 

More information is available via their blog: http://insidechannelmeter.wordpress.com/2011/05/25/advice-to-developers-on-managing-mysql/

So the long and short of it...... Way to go open source !