Skip to main content

Creating a date dimension in dbt and Snowflake

A lot of the data that I am working with doesn't lend itself to creating complex star schemas (more on data modelling in a later post) however I want to try and at least want to go to some effort. The one thing that pretty much all my tables have in common is date, so I want to build a date dimension. My strength training happens on a day, my running and walking, parkrun and hopefully the weather data I intend to integrate will have a date on it. This means if I have a date dimension and I aggregate my various facts at date level I can use them together in a single visualisation. 

There are various methods I could have used to create the date dimension and I went with using the dbt spine, this was really simple using the utilities: 


Used the date functions in Snowflake to extract relevant fields such as year, month, day in words and year month etc. 

Comments

Popular posts from this blog

Zapier

As much as I have enjoyed using Keboola there are some connections that it doesn't have or that just haven't worked for one reason or another. I actually came across Zapier as a solution for bringing in e-mails from parkrun to load my results every week. Honestly I have not found it to be as robust as Keboola but that might just be me archiving my e-mails before it completes it 15 minute poll.  The second use case I am working on is the pulling in Strava data, for a fitness dashboard the fact it has a built in connector for Strava is great, though I am worried given the activities I do that I might reach the limit.  I won't go into details on how to set things up but you can setup 5 Zaps that can run for a combined 100 runs during a month for free.  In my data platform / solution I am using Zaps to load harder to get / automate data. It doesn't add much from a technical point of view as it is just signing into a few account to get the data into Google Sheets for downstr...

Keboola Flows

Really finding Keboola was the thing that kickstarted this project otherwise I would be trying to build custom code on a python cloud server and building everything from scratch.  In Keboola you build you data sources and destinations using connection details which is fairly simple and something I will likely cover in another post, same goes for transformations etc. Here though I am going to discuss Flows, this is where you bring everything together. On my free account there are some limitations.  My easiest flow is very basic:  Pull parkrun results e-mail from Gmail to Google Sheets (actually done by Zap not Keboola).  Keboola will, as often as I like, in this case once a week, pull the data from the sheet into its storage.  It will then transfer this to the target database. Currently I have this setup to be MySQL database but I can and might expand that to the Snowflake instance within Keboola.  I then, outside of Keboola, connect to the MySQL database f...

Generative AI News - 03/03/2024

Here are some of the latest news highlights related to Generative AI from this week: Venus Williams and Brain.ai : Tennis legend Venus Williams has ventured into interior design with Brain.ai , a new generative AI-powered platform 1 . It’s fascinating to see how generative AI is being integrated into various domains beyond smartphones. Google’s Gemini AI Controversy : Google’s chatbot, Gemini , faced controversy due to its image generator mistakenly placing people of colour in Nazi-era uniforms. This incident highlights the importance of responsible AI development 2 . Apple’s Focus on GenAI : Apple CEO Tim Cook announced that the company will “break new ground” in generative AI this year 2 . Keep an eye out for exciting developments from Apple. Morph Studio for Filmmaking : Morph Studio introduces an AI filmmaking platform that allows weaving Stability AI-generated video clips into movies 2 . A creative application of generative AI! Adobe’s Project Music : Adobe unveiled Project Music...