What’s maybe more exciting, though, is something Databricks calls Project Lightspeed, which the company describes as the next generation of the Spark streaming engine.

First created as part of a research project at UC Berkeley AMPLab, Spark is an open source project in the big data space, built for sophisticated analytics, speed, and ease of use. It unifies critical ...

The June update to Apache Spark brought support for R, a significant enhancement that opens the big data platform to a large audience of new potential users. Support for R in Spark 1.4 also gives ...

Databricks/Spark on the other hand has had support for Python for a while now, which may help explain what we perceive as a broad differentiation between the two platforms: Flink is used more as a ...

Apache Spark 3.0 is now here, and it’s bringing a host of enhancements across its diverse range of capabilities. The headliner is an big bump in performance for the SQL engine and better coverage of ...

Databricks Cloud will provide Spark-based streaming analysis as a service Taking on Google, Databricks plans to offer its own cloud service for analyzing live data streams, one based on the Apache ...

Still, Databricks’ announcements today failed to address its in-memory data processing capabilities, which Mueller said was Spark’s biggest strength but also its biggest weakness.