Skip to main content

Keboola Flows

Really finding Keboola was the thing that kickstarted this project otherwise I would be trying to build custom code on a python cloud server and building everything from scratch. 

In Keboola you build you data sources and destinations using connection details which is fairly simple and something I will likely cover in another post, same goes for transformations etc. Here though I am going to discuss Flows, this is where you bring everything together. On my free account there are some limitations. 

My easiest flow is very basic: 

  • Pull parkrun results e-mail from Gmail to Google Sheets (actually done by Zap not Keboola). 
  • Keboola will, as often as I like, in this case once a week, pull the data from the sheet into its storage. 
  • It will then transfer this to the target database. Currently I have this setup to be MySQL database but I can and might expand that to the Snowflake instance within Keboola. 
  • I then, outside of Keboola, connect to the MySQL database from Google Data Studio and make some visualisations. 
Within Keboola flows you have several tabs. The builder tab where you configure your flow. The All Runs tab to look at the logs of Flow Runs, Notifications where you can configure e-mail notifications for various results of the Flow (Success, Error etc.) and the versions tab where you can look at the history of the flow. 

You can see many of these steps and the basic config in the below gif. 


The end result of this flow is the e-mail date, subject and body being passed into the MySQL database. I then do some data cleansing on this (post to come) and then visualise (currently poorly) in Google Data Studio. I intend to do another post on comparing this very simple data set by visualising in GDS, Power BI and Retool. 


Labels:

Comments

Post a Comment

Popular posts from this blog

Generative AI News - 03/03/2024

Here are some of the latest news highlights related to Generative AI from this week: Venus Williams and Brain.ai : Tennis legend Venus Williams has ventured into interior design with Brain.ai , a new generative AI-powered platform 1 . It’s fascinating to see how generative AI is being integrated into various domains beyond smartphones. Google’s Gemini AI Controversy : Google’s chatbot, Gemini , faced controversy due to its image generator mistakenly placing people of colour in Nazi-era uniforms. This incident highlights the importance of responsible AI development 2 . Apple’s Focus on GenAI : Apple CEO Tim Cook announced that the company will “break new ground” in generative AI this year 2 . Keep an eye out for exciting developments from Apple. Morph Studio for Filmmaking : Morph Studio introduces an AI filmmaking platform that allows weaving Stability AI-generated video clips into movies 2 . A creative application of generative AI! Adobe’s Project Music : Adobe unveiled Project Music...
Post a Comment
Read more

Zoho Analytics

Have I finally found my BI Tool, one that lets me import data from Snowflake and share it for free? I know, no sooner have I posted about how hard it was to find a tool that could do anything from Snowflake than I come across Zoho. You can check out my dashboard on the following page . Below is a diagram that outlines the processes I have used to obtain this data. In summary my parkrun e-mail is pushed to Google Sheets every week by Zapier and Forms I submit every day are used to track the strength training I do. Keboola is then used to ingest this data into MySQL and or Snowflake where I then use views or the built in transformation processes in Keboola to shift the data into a format for reporting. Google Data Studio then connects to MySQL and Zoho to Snowflake to visualise the data. 
Post a Comment
Read more