Really finding Keboola was the thing that kickstarted this project otherwise I would be trying to build custom code on a python cloud server and building everything from scratch.
In Keboola you build you data sources and destinations using connection details which is fairly simple and something I will likely cover in another post, same goes for transformations etc. Here though I am going to discuss Flows, this is where you bring everything together. On my free account there are some limitations.
My easiest flow is very basic:
- Pull parkrun results e-mail from Gmail to Google Sheets (actually done by Zap not Keboola).
- Keboola will, as often as I like, in this case once a week, pull the data from the sheet into its storage.
- It will then transfer this to the target database. Currently I have this setup to be MySQL database but I can and might expand that to the Snowflake instance within Keboola.
- I then, outside of Keboola, connect to the MySQL database from Google Data Studio and make some visualisations.
Within Keboola flows you have several tabs. The builder tab where you configure your flow. The All Runs tab to look at the logs of Flow Runs, Notifications where you can configure e-mail notifications for various results of the Flow (Success, Error etc.) and the versions tab where you can look at the history of the flow.
You can see many of these steps and the basic config in the below gif.
Comments
Post a Comment