dbt: Programmatic Invocation via dbtRunner

Introduction dbt is a great tool for building & organising your ELT data pipelines. When deploying dbt yourself you can invoke dbt either through dbt core cli or through Python via dbtRunner. I will give you an example template on how to use the latter. You can find the full example in the Full Example section. Note: This example was build on dbt-core==1.8.3. dbtRunner may be subject to breaking changes so there’s no guarantee the provided code works as is with other dbt versions....

August 6, 2024 · 9 min · 1823 words · Andreas Lay

Hackerfluff: A Hackernews Reader built with Flutter

A Hackernews Client In an attempt to learn some frontend / mobile development I’ve decided to give Flutter a try. Flutter is cross-platform framework developed by Google which lets us build & deploy for web (iOS & Android), web and desktop from the same code base. I’m an avid Hacker News reader, they provide an API to query stories and comments, so I decided to build a Hackernews client app....

May 22, 2024 · 4 min · 789 words · Andreas Lay

Securing any App with Oauth2Proxy

Introduction When you deploy applications you might want to protect them behind a login. If you’re deploying multiple applications it might not be feasible to add authentication for each deployment separately. Here I’ll show how to set up a load balancer with nginx and oauth2proxy using Keycloak to secure any app. Run the example You can find a working example of running a load balancer with authentication in this repository. You can use docker compose to run the example:...

November 23, 2023 · 4 min · 826 words · Andreas Lay

A Primer on SARIMAX

A while ago I created a notebook with an introduction to time series analysis. Here is this notebook as a Gist: Generate a synthetic time series with cycles, trend (random walk) and noise components Look at some descriptive statistics (e.g. autocorrelations) Model the synthetic data with a SARIMA model Working with synthetic data first forces you to be explicit about your assumptions and is great for debugging: Unlike real data, as you know the true process the synthetic data follows you can validate your estimates easily against the “true” values....

November 21, 2023 · 1 min · 109 words · Andreas Lay

Mac Quality-of-Life Tooling

Here’s a short list of Mac apps which make your life a little bit easier: AltTab: Use windows “alt-tab” shortcut on Mac to switch windows Rectangle: Resizing windows using keyboard shortcuts Itsycal: A nice calendar for your menu bar Flycut: Clipboard manager for developers

November 20, 2023 · 1 min · 44 words · Andreas Lay