过去十年，数据工程的主线，是 Modern Data Stack 对传统数仓体系的一次拆解与重组。 我们把数据采集从数据库里拆出来，形成了 Data Ingestion，用 FiveTran、Airbyte、Apache SeaTunnel 来解决 ELT / CDC / Reverse ETL； 把计算从存储里拆出来，形成了 Snowflake、Databricks、Iceberg ...

Cloudera宣布，通过与NVIDIA合作，已在Cloudera Data Engineering中为Apache Spark 4.1引入原生GPU加速功能。该功能将作为Cloudera Anywhere ...

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources. David Chisnall discusses how the CHERI ...

Though the AI era conjures a futuristic, tech-advanced image of the present, AI fundamentally depends on the same data standards that have been around forever. These data standards—such as being clean ...

A senior data engineer's honest first impressions after a Palantir Foundry bootcamp, including what it does better, what it ...

Credit: Image generated by VentureBeat with FLUX-pro-1.1-ultra A quiet revolution is reshaping enterprise data engineering. Python developers are building production data pipelines in minutes using ...

The death of SQL has been greatly exaggerated. With all apologies to Mark Twain, the demise of Structured Query Language (SQL), having been forecasted for a while, is far from a done deal. In fact, ...

Cloudera, the only company bringing AI to data anywhere, today announced native GPU acceleration for Apache Spark 4.1 in Cloudera Data Engineering, enabled by the NVIDIA CUDA-X library, cuDF. The ...

One of the critical decisions facing companies embarking on big data projects is which database to use, and often that decision swings between SQL and NoSQL. SQL has the impressive track record, the ...

A report published by Start Us Insights depicts the global data engineering landscape currently featuring 1,554 startups, with innovative companies like dEX from Brazil, PurpleCube AI from the USA, ...