Skip to main content

dbt training - the next part

 So I enjoyed the first dbt training enough to give the next bunch of training a go. The first part of this training introduces you to jinja and using it to, for example, loop through creating case statements to pivot out rows to columns. 

Jinja Code - 

{%- set activities = ['Pull_Up', 'Crunch','Push_Up','Plank_Time','Dumbbells','Box_Jump'] -%}
select date_done,
{% for activity in activities -%}
sum(case when replace(description,' ','_') = '{{ activity }}' then act_count else 0 end)
as {{ activity }}_count
{%- if not loop.last -%}
,
{% endif -%}
{%- endfor %}
from {{ source('exercise_data','V_EXERCISE_STAR') }}
group by 1


I can see there are lots of really useful purposes for this and am hoping that in I will find out later that rather than using a hard coded list you can use a query to generate the list of entries to loop through. This would be a great feature, one of the limitations of a lot of SQL systems is you can't pivot on an unknown list of values. Now onto learning about macros and packages in dbt. 

Generates SQL Code -  

select date_done,
sum(case when replace(description,' ','_') = 'Pull_Up' then act_count else 0 end)
as Pull_Up_count,
sum(case when replace(description,' ','_') = 'Crunch' then act_count else 0 end)
as Crunch_count,
sum(case when replace(description,' ','_') = 'Push_Up' then act_count else 0 end)
as Push_Up_count,
sum(case when replace(description,' ','_') = 'Plank_Time' then act_count else 0 end)
as Plank_Time_count,
sum(case when replace(description,' ','_') = 'Dumbbells' then act_count else 0 end)
as Dumbbells_count,
sum(case when replace(description,' ','_') = 'Box_Jump' then act_count else 0 end)
as Box_Jump_count
from KEBOOLA_7127.WORKSPACE_15661914.V_EXERCISE_STAR
group by 1

Comments

Popular posts from this blog

AWS training cloud academy free course

One of the things I like about this course are the instructors are really clear but also that it provides free labs that allow you to actually sign into AWS and perform some actions to actually create and do things without worrying that you are going to incur a cost.  Today I complete one of the hands on labs.  This was to create a lambda function, in this case it was a very basic python script that was searching a website for a keyword. I then placed this into a schedule and used cloudwatch to create a dashboard that monitored the running of this function. Overall it was a very simple use case but it was also a very simple process to setup.  I don't have much to add to this other than it is well worth signing up to cloud academy for the free training if nothing else, I am tempted, once i have done some more training, to give the paid for option a go to get the full sandboxes. 

AI News

Here’s a concise roundup of the latest AI news from the past couple of days: AI Technology: Friend or Foe? Researchers and experts continue to debate the impact of artificial intelligence. Is it a boon or a threat? The discussion ranges from AI ethics to its potential in various fields. Read more here . 5 Ways Artificial Intelligence Will Change the World by 2050 Futurists predict that AI will revolutionize our lives in the coming decades. From healthcare to transportation, AI is set to transform industries. Explore the possibilities here . How AI Will Transform Businesses in 2023 Business leaders are embracing AI to enhance efficiency, decision-making, and customer experiences. Stay updated on the latest AI trends in the corporate world here . China’s High-Level AI Modules China is pushing the boundaries of AI with modular next-generation systems. These high-level AI technologies promise breakthroughs in fields like robotics, healthcare, and smart cities. Learn more here . The Future ...

Data Cleansing View in Snowflake

For part of one of my free ETLs I am using Zapps to transfer e-mails from Google Sheets and then Keboola to transfer the sheets into my Snowflake database. I am familiar with string searches and cleansing in Oracle and using python but have not had the chance to do this in Snowflake. I wanted to give it a go as a proof of concept if nothing else. There were some difference in functions between Oracle and Snowflake, no INSTR and using POSITION instead and some difference in working with dates / timestamps but overall it was very similar.  The code below is what I ended up using:  I think want to use this to create some overview graphics to allow me to track the success or failure of my ETLs. Assuming the aspects of Retool remain free you can see how much ETL is going on this link .  In case things aren't working, here is a table of the output I am producing.