Recent searches
Search options
#airflow
0 posts0 participants0 posts today
Just caught up with the recent Delta Lake webinar,
> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust
Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. #ApacheAirflow tasks
Airflow for Beginners 
A short tutorial by Sunjana Ramana for building an ETL data pipeline using Apache Airflow:
https://www.youtube.com/watch?v=3xyoM28B40Y
I think #Dave2D is a bit too excited about an #ionization-based #SolidStateCooler aka. "#fanless #fan"...
- OFC it'll not be able to compete in terms of #price, #StaticPressure and #Airflow with #fans, and I wounder how well it'll fare with #dust and #hairs but I can see why he likes the concept.
I just think it'll end up in #nieche devices that want to have the option to additionall boost power on a normally #PassiveCooling system to make it a #silent system without relying on convection alone.
- I'd guess we'd rather see some #iPadPro or #MacBookPro than a #ThinkPad or #Framework13 with it, espechally since it'll require a #CatalyticConverter to reduce it's #ozone emissions...
Cuz #AirChemistry and #Physics ain't something #Ventiva can innovate themselves out of...
Curious about how #Airflow schedules workflows?
This blog dives into the technical details of Airflow’s scheduling mechanism, breaking down how it orchestrates data pipelines with precision and efficiency.
Perfect for #DataEngineering: https://ter.li/rxiopj
Quoting an Airflow log output:
scheduler | [2024-09-07T17:51:43.325+0200] {workday.py:41} WARNING - Could not import pandas. Holidays will not be considered.
Stay strong, dear developer!
Portable Multi-SDR Rig Keeps Your Radios Cool https://hackaday.com/2024/09/03/portable-multi-sdr-rig-keeps-your-radios-cool/ #software-definedradio #3Dprintedenclosure #RadioHacks #aircooling #airflow #RTL-SDR
Hackaday · Portable Multi-SDR Rig Keeps Your Radios CoolWith as cheap and versatile as RTL-SDR devices are, it’s a good idea to have a couple of them on hand for some rainy day hacking. In fact, depending on what signals you’re trying to sni…
Portable Multi-SDR Rig Keeps Your Radios Cool - With as cheap and versatile as RTL-SDR devices are, it’s a good idea to have a cou... - https://hackaday.com/2024/09/03/portable-multi-sdr-rig-keeps-your-radios-cool/ #software-definedradio #3dprintedenclosure #radiohacks #aircooling #airflow #rtl-sdr
Hackaday · Portable Multi-SDR Rig Keeps Your Radios CoolWith as cheap and versatile as RTL-SDR devices are, it’s a good idea to have a couple of them on hand for some rainy day hacking. In fact, depending on what signals you’re trying to sni…
The #Zhukovsky #Aerofoil (sometimes transliterated as #Joukowsky from #Russian), is a 2D model of #streamlined #Airflow past a #wing. It uses #ComplexVariable and is an #AnalyticFunction (i.e. #Differentiable everywhere, save at isolated #Singularities). Take a circle in the #ComplexPlane which is not quite centred at the #origin but passes through the #coordinate (1,0) or (z=1+0i).
GIF
Hi #Python experts / #pythonista (s) - I am looking for recommendations for libraries to model and automate/orchestrate a business process. I need something lightweight that doesn't require a backend (like #airflow) but I need to be able to generate a #DAG to #visualize the business process. TIA.
#workflow #business #flowchart #businessprocess #orchestration
I saw this on #iFDesign and found it brilliant : « Dynamic Airflow » is a type of hinge for #laptop who permits "the displacement of the heat module to an external position, simultaneously enhancing #airflow" when opening the screen 
› https://ifdesign.com/en/winner-ranking/project/dynamic-airflow/613680
Check this out!
#DAGMonitor is an iOS app for data engineers that helps monitor and control DAGs on your server. With help of the app, data engineers can access server jobs, monitor task instances, rerun them and read through task logs. Simply specify the username and password in the app settings and connect to your company's network (if your server is available only from the corporate WiFi or VPN).
FEATURES AVAILABLE IN VERSION 1.0:
• Connect to server via API (simple user/password authentication supported)
• List DAGs and get info about paused ones
• Access DAG runs and monitor statuses
• List task instances for DAG run
• Clear status for a task instance
• Access task instance logs
App StoreDAG MonitorDAG Monitor is an app for data engineers that helps monitor and control DAGs on your server. With help of the app, data engineers can access server jobs, monitor task instances, rerun them and read through task logs.
FEATURES:
• Connect to server via API
• List DAGs and get info about paused ones
•…
I'm currently evaluating various workflow engines (#Airflow et. al.) I'm pretty amazed how #Temporal (fork of #Cadence by #Uber) allows writing simple, imperative workflows. When your workflow (implemented as a long-running function) crashes, it's state is persisted seamlessly and resumed on other node
- In #Airflow my `response_check` callback does not run when `SimpleHttpOperator` receives HTTP 4xx
- Yes, it's bypassed for non 2xx codes. In order to run `response_check` you must configure `check_response` to `False`
- WAT?
- What?
```
task = SimpleHttpOperator(
extra_options={'check_response': False},
response_check=lambda ..., # now it runs
# ...
)
```