SQLPass meeting 21 september 2010

I joined a session at the dutch chapter of SQLpass and it was about gamechangers. Due to recent developments (SSD, fusion IO) in hardware and databasetechnology (NoSQL, columnar databases) innovations are going pretty fast. So what does this mean for the work of a DBA, SQL Developer, datawarehouse architect or anyone who is working with databases? The session had two main parts : a presentation video of DR. David DeWitt (Microsoft Data and Storage) from PASS Summit  2009 and the next part was a discussion about what does the recent developments mean for the current and future job activities of the (SQL Server) database specialist.

The first part of this night was a videopresentation of Dr David deWitt at PASS Summit 2009 and was about the developments in hardware over the past 30 years and he analyzed that the current disks are 10000x larger than 30 years ago, memory is 1000 x larger, CPUs are 2000x faster and CPU Batches are 1000x faster. As he explained further, CPU has become 1000x faster but IO has only become 10x faster and SSD seems to be the only real hope for getting things done faster. Also he made a comparison between the transferrate of data 30 years ago and nowadays and the outcome was that the disks are 150 x has become relatively slower than 30 years ago!! So improvements in CPU and memory increased more than improvements on disks.

After that a lot of technical stuff with L1, L2 and memory caching he explained that the column store is, in theory, 7 times faster than row stored based databases. But there’s a tradeoff, updating is slower than row based databases. Conclusion was that column based databases reduces the amount of disk I/O required  to execute a query and improves CPU performance by reducing memory stalls.  Best practice is not the use a column store database for operational purposes. He believed that due to hardwaretrends more specialization will happen in the future.

After this a discussion started about SSD’s, fusion IO’s, IOPS, virtual file stats, use SSD for tempdb’s, listing pending I/O’s. Also there was a talk about integrated memory of memory and storage. This is an interesting development because I learned from the first books about computers that the Von Neuman architecture always is based on a cpu, an in- and external memory. This paradigm will change in the future. No more harddisks? What would this mean? Are things going so fast that there’s less need for work done by ICT professionals, like architecture development, DBA and performance tuning, etc? Is that the future? Or will volumes of data grow that fast that there will always a need for ICT professionals? Time will learn…

Another viewpoint is that 10 year ago the world was much easier. You had a database and VB6 application and that was it. But nowadays we have all kind of new features in products, complex environments, more data, more (different) user needs. As Dr David deWitt states more specialized products will be on the market in the future and therefore more specialization of ICT-professionals is inevitably.

So the title of the evening was 'Gamechangers'. What is a gamechanger? I think that you should define what a gamechanger is. As we look at the past what are typical gamechangers: the IBM desktop, internet, mobile devices are typical gamechangers. Also you could define gamechangers as a revolution, a turnaround. In this discussion, i didn't hear a gamechanger. The stuff i heard were more a evolutions: the column based database is specialized database (evolution) and the SSD disk is an evolution of the external memory. Things (talked about at that SQL Pass evening) don't change that much, only are going faster or are more specialized.

Greetz,

Hennie