When you want to test functions that deal with date, it could be harder than regular tests, especially if you deal with
datetime.now, but workarounds exists !
To test processing and functions that use time functions in Python (i.e.
datetime.now), you have two strategies: you set or you mock.
Most of the time you can mock (use a fake object that “mimic the behavior of real objects”) datetime, but sometimes you can not mock because libraries you use in your code check types (ie:
For instance if you want to test a function
get_beginning_time_window that returns seconds since the beginning of a time window (defined by a string, ie “5 hours”). This compute is defined as: if you ask for a time window of 5 hours at 16h34 it must return
(5*60 + 34) *60 (so seconds since 11h), so for “5 hours” this function could return 18000 to 21600. …
If you develop in the cloud or on a shared server, you might have experienced the beauty of working with your teamate relying on the same codebase and the same database! Actually, when working on the same cloud environment with other people, it happens that certain resources (files, tables…) are shared between several teammates. This means you can sometimes find yourself blocked because other people are working on the same resource. And it can become frustrating when a file/table you needed is overwritten, moved or deleted…
If you want to do most of your Airflow’s work without relying on a shared workspace and avoiding latency, use an Airflow Docker container and run syntax and unit tests in it. …