Skip to main content

Python Pipeline API to MySQL to Snowflake

I decided that I liked doing some of my coding in python, even if I have to manually kick it off at the moment I might go with the safe option of a $5 a month python anywhere package to run on a schedule in the cloud or I could put in as an AWS lambda function or in Azure but I don't want to accidentally rack up a bill so might wait until I am further into training on them. 

So in this code I have:

  • Used dotenv to store all parameters and passwords as environment variables so I can post my scrips without modification and store them in git. (with the env file set to gitignore). 
  • Retrieved the values above and called the weather API. Flattened the json to get all the columns. Put the new rows into the table in MySQL. 
  • Retrieved the table from MySQL and done a drop and replace into Snowflake. 

My Code: 

The table: 



Comments