It's happening again. The Apache Software Foundation, which promotes open source software projects, is brewing yet another technology that could redefine corporate and cloud data centers.
Among the Apache projects worth tracking: Apache Hadoop (surely you know that one), Apache Spark (get to know this one fast) and Apache Mesos (get this on your radar). Here's a closer look at each effort.
1. Apache Hadoop: So, you need a big data strategy. Quite a few businesses now leverage Apache Hadoop to build low-cost storage grids. Popular commercial Hadoop distributions include Cloudera, Hortonworks and MapR (among others). Those Hadoop grids, once built, become data lakes that pump information into big data analytics applications. It sounds simple, but setting up Hadoop on-premises can be complicated. The result: Some folks think the Hadoop industry will grow more slowly than originally expected, though Hortonworks VP Shaun Connolly sounds upbeat. That's why an ecosystem of experts like BlueData is starting to emerge. Plus, cloud-based Hadoop services are coming into focus.
2. Apache Spark: Some pundits think Spark competes with Hadoop. Others, such as Cloudera Chief Technologist Eli Collins, think Spark works effectively with Hadoop. Either way, here's what you need to know. First, Spark is more of a real-time technology -- allowing data scientists to perform real-time big data analytics on near real-time data. Second, IBM is betting the house on Spark -- asserting that Spark could become a big data platform the way Linux became a data center and cloud operating system.
3. Apache Mesos: Mesos allows you to "program against your datacenter like it's a single pool of resources." Hmmm... That sounds a lot like VMware's software-defined data center vision, and so many other software-defined mission statements that we hear from traditional IT vendors. But Mesos is open source and it's starting to gain real momentum. Key adopters include Verizon, which now uses Mesophere DCOS as a "nationwide platform for data center service orchestration." Poke around and you'll find more Apache Mesos adopter stories popping up -- especially in the social media industry.
I recently interviewed IBM's top Spark pundit and will be sharing more thoughts over on Information Management in the days aheads. In the meantime, keep an eye on emerging projects at the Apache Software Foundation. When it comes to finding out "what's next" in corporate IT, the Apache organization usually has some intriguing answers.
Subscribe: Want to receive our blog headlines in your inbox each business day (except Fridays, when we're off the grid)? Then subscribe to our enewsletter. Thanks to those who already have.