After an incredible 13 year career, Dan Delorey left Google to become the VP of Data at SoFi. It’s a huge switch, and — fortunately for us — Dan shared his story with the Data Engineering Podcast. Here I want to share my notes, starting with Dan as a Google Software Engineer who became an Engineering Director for BigQuery, and then moved into a VP role at a regulated financial company. — This is a fascinating conversation, as it moves from a deep focus on query execution and data storage — and then moves into business problems with data at SoFi (like “finding data, making sure data is reliable, monitoring SLAs about data delivery, answering business user questions”).

Now Facebook Prophet runs inside Snowflake — no need for external functions or dependencies. Let’s see how. — Previously I showed how to run Facebook Prophet as an external function, and then I wrote about running pmdarima inside Snowflake as a Python UDF — thanks to Snowpark and Anaconda. Today I’m happy to announce that Facebook Prophet runs inside Snowflake — without any additional hassles or compilation steps…

Shruti Sekaran is one of the amazing interns I met at Snowflake during the summer of 2022. I was immediately impressed by her goal: Improving the performance of some specific SQL patterns in Snowflake by “eliminating redundant semi-joins and anti-joins from queries”. Check this post for some SQL optimization tricks, and why you might not need them. — SQL quiz: Optimize this Before we go deeper, check the following query. It scans 6 billion rows, while performing two semi-joins with sub-queries that go over 1.4 billion rows each. How would you optimize it?

Snowflake customers can now instantly access premium datasets, while providers choose how to charge for them (like monthly or per query). Meanwhile Snowflake takes care of making the magic happen in the backend. Let’s explore this new feature, now available for Snowflake customers in 33 countries. — The new “Paid” datasets Let’s start with an example: IPinfo has datasets ready to help you translate IPv4 and IPv6 addresses into real-world data like a country, a mobile carrier, or a company. IPinfo already has these datasets in the Snowflake Marketplace, and you can find them with a quick search:

“I want a table with all the Holidays” is a typical request in data analytics — but figuring this out is not as easy as it sounds. For which country, for which sub-region, for which timeframe — are only the initial questions. Then come some technical challenges, that we’ll figure out in this post with Snowflake and Python — including how to update the Anaconda pre-installed packages in Snowpark. —

Let’s find out the easiest way to get ARIMA predictions in Snowflake with a Python UDF — available right now in Public Preview. Also find here why December 2025 will be a significant date for Snowflake’s presence in Stack Overflow. — 2022–09 update Facebook Prophet now runs inside Snowflake — check the post for more. Facebook Prophet forecasts running in Snowflake with Snowpark Now Facebook Prophet runs inside Snowflake — no need for external functions. Let’s see how.hoffa.medium.com

Where Tal tells us why he chose to join Snowflake after Google, and what’s coming next. — It was exciting to see Tal Shaked joining Snowflake after more than 16 years at Google. Something that’s fascinating about Tal’s life is that, in 2004, in those days when the world was learning about MapReduce, he was already doing machine learning at Google. So I called…

Counting unique snowflakes is fun. Thanks to Google open sourcing ZetaSketch and Snowflake’s ability to run native Java UDFs, we can achieve sketch compatibility for approximate counts with other ecosystem players. — In 2019 Google open-sourced ZetaSketch, a “collection of libraries for single-pass, distributed, approximate aggregation and sketching algorithms”. This has allowed other projects to enable compatibility for approximate counts and partial results:

This pattern will allow you to create high-traffic apps available to the world, hosted by Streamlit Cloud. Play and prepare your data with Snowflake and Streamlit, and then serve the results in production out of static Parquet files. Snowflake acquired Streamlit for a good reason: Streamlit adds magic to the…