Published inTDS ArchiveMastering ExternalTaskSensor in Apache Airflow: How to Calculate Execution DeltaExternal Task Sensors stop bad data from trickling downstream in a data pipeline. Leverage them to create a reliable data infrastructure.May 8, 20236May 8, 20236
Published inTDS ArchiveThe Art of Speeding Up Python LoopThere is no “best” looping technique in Python, only the most suitable.Oct 31, 20221Oct 31, 20221
Published inTDS Archive14 Best Practices to Tune BigQuery SQL PerformanceWith big data, querying is no longer just about writing the “correct” syntax, it needs to be cost-effective and fast, too. Here is how…May 3, 202210May 3, 202210
Published inTDS ArchiveShannon Information: Discovering Atoms of CommunicationPhysical objects have atoms, information has bits. Claude Shannon believes that information, although intangible, can be quantified…Mar 21, 20223Mar 21, 20223
Published inTDS ArchivePrincipal Component Analysis (PCA) Explained Visually with Zero MathPrincipal Component Analysis (PCA) is an indispensable tool for visualization and dimensionality reduction for data science but is often…Feb 3, 202217Feb 3, 202217
