What's next in the IT market? At the enterprise level, I'm starting to think the answer potentially involves Apache Spark -- an open source platform for "lightening-fast cluster computing."
Indeed, Spark Summit this week in San Francisco is expected to attract about 2,000 attendees -- roughly twice last year's audience. Big companies -- from IBM to Intel -- will be there in full force. In fact, IBM has spent the past two weeks leaking clues about its Spark strategy -- which includes a new research center in the San Francisco area.
From established IT giants to tech startups, just about everyone is trying to get ahead of the big data curve -- upon which businesses are trying to more effectively gather, store, manage, optimize, protect and monetize data. Much of the industry buzz has so far included Hadoop -- an open source storage cluster technology.
Names like Cloudera, Hortonworks and MapR have been driving the Hadoop market. Hortonworks VP of Corporate Strategy Shaun Connolly explained some Hadoop market trends in this podcast. Gartner has questioned Hadoop adoption rates, even as a growing number of Hadoop enterprise customers sing its praises.
Sparking the Next Revolution
Now along comes Spark -- which sounds similar to Hadoop... but could be quite different. Most of the major big data players see opportunities for Spark and Hadoop to live together, much in the way that VMware, OpenStack and Docker each have their roles in the virtualization, cloud and container worlds.
Still, the Spark noise is going to be very loud this week. And the company to watch -- other than IBM -- is Databricks. Similar to how Red Hat emerged as the corporate Linux leader, Databricks wants to be the corporate Spark leader. And in fact, Databricks' founders are Spark's inventors...
So what surprises will Spark Summit deliver? I expect quite a few from IBM, Databricks and others. Stay tuned for more details as the summit unfolds starting today. Oh, and one other thing. Keep a close eye on Blue Data Software -- another startup that could have plenty to say about how you deploy Spark and Hadoop clusters on-premises...
Subscribe: Want to receive our blog headlines in your inbox each business day? Then subscribe to our enewsletter. Thanks to those who already have.