ETL.
Every dev3lop article on ETL, newest first.
ET1 Concat Node
Bring your columns together as one with the Concat Node in ET1. This node is similar to concat() in Excel and allows you to easily bring more than 1...
ET1 Constant Node
The Constant Node creates a constant value per row in your data pipeline. This node is extremely handy when transforming data in your ETL processes. The...
ET1 Find/Replace Node
Automatically finding and replacing data is possible using the Find/Replace Node! Find and replace works inside of sentences, words, numbers, and anywhere...
ET1 Manual Table Node
Create a table manually using the Manual Table Node. Manual Table node falls under the (https://dev3lop.com/et1-data-input-node-overview/) category. Built...
ET1 CSV Input Node
The CSV Input Node, what a classic, flat files living on your computer can be consumed and the data can be extracted here in ET1. CSV is a common file...
ET1 Github CSV Node
ET1's Github CSV Node is designed to help end users extract data from Github CSV URLs which are in public repositories. A public repository on Github is...
ET1 JSON Input Node
When extracting data from a JSON file, try the JSON Input Node. JSON (JavaScript Object Notation) is a common data source. With ET1's JSON Input Node you...
ET1 Trim/Normalize Node
Trim/Normalize Node is built to help you quickly clean your data pipelines and like the (https://dev3lop.com/et1s-column-renamer/), built to make data...
ET1 Joiner Node
On your magic quest to join data? We call it the Joiner node. A simple joining solution that helps people join data at a row level. In ET1, Joiner is...
ET1 Group By Overview
Eager to group data? The Group By feature can be found in the Aggregation Node. Add the aggregation node to the canvas and send data downstream to this...
ET1 Measure Filter Node
When you have numbers, you have a need for a Measure Filter Node. Numbers are here, lets talk about it. Filtering with a number or decimal is straight...
ET1 Split Node
The Split node lets ET1 user split one or more column into multiple columns. This particular node is great for the times you have multiple rows of data...
ET1 Unique Filter Node
The Unique Filter Node or Unique Tool finds unique values per row in your data pipelines, or allows people to quickly review duplicates only. Plus, you...
Append Fields with ET1's Joiner
Seeking to append fields like the Alteryx Desktop software? The (https://dev3lop.com/et1s-joiner-node/) and...
ET1 Duplicate Columns Node
Dealing with duplicate columns? This particular node is designed to remove similarly named column headers. If "State"="State" then we remove the last...
About ET1, A Tool to Help Uncomplicate Data
It is undeniable that data can be overwhelming, and those who work with it daily are adept at navigating this complexity. ET1 is designed to assist...
ET1 Data Combination Tools
Are you combining the data? We have you covered. ET1 has all the right tools. The Three Musketeers of Data Combination 1. 🤝 Join (The Matchmaker) -...
ET1 Basic Training
ET1 helps you extract, transform, and load data in a single user-friendly canvas. Data automation in a single canvas. Each node is simple and easy to use...
ET1's Data Input Node Overview
CSV, JSON, and Public CSV endpoints or manual tables. These help you kick start your data pipeline. Once your data comes into the data input, it begins...
Filtering Nodes in ET1
The filtering nodes help you reduce the number of rows, drill into the exact information needed, and create a data set that will add value VS confuse your...
Adapter Pattern: Converting Formats on the Fly
In today's rapidly evolving digital landscape, data integration poses an ongoing challenge for enterprises striving for streamlined operations and...
Data on a Shoestring: Open Source vs Enterprise Pipeline Costs
Every organization aims to become data-driven, but not every organization enjoys unlimited resources to achieve that vision. Leaders tasked with managing...
Graceful Degradation: Surviving When Everything Goes Wrong in Batch Jobs
Picture this: your data-driven enterprise relies heavily on nightly batch processing to power critical business decisions, but one evening, disaster...
Multimedia Pipelines: Extracting Metadata from Binary Blobs
In a digital-first world, multimedia is a core foundation of nearly every business-savvy decision—whether you're streaming high-definition videos,...
Processing Dirty CSVs: Handling Malformed Headers and Encoding Issues
In today's data-driven landscape, organizations rely on structured data files such as CSVs (Comma Separated Values) to unlock crucial insights and foster...
Building Idempotent Data Processors: Why Your Pipeline Shouldn't Fear Reruns
Picture this: it's 2 AM and you receive an alert that your critical data pipeline has failed mid-run. You dive out of bed, heart racing, wondering how...
Taming the Wild West of Nested JSON: Advanced Flattening Techniques
In today's data-intensive world, dealing with nested JSON structures is like navigating the Wild West of data management: vast opportunities, but equally...
How to Transition from Traditional ETL to Modern Data Engineering
Businesses today live and breathe data, needing access not just to raw information but sophisticated insights that strategically empower decisions....
Data Pipeline Parameterization for Multi-Tenant Processing
In an age where adaptability, scalability, and smart analytics are critical for growth, businesses serving multiple clients—each with unique data...
Handling Sensitive Data in ETL Processes: Masking and Tokenization
In an age where data has become the critical backbone fueling innovation, companies grapple daily with the significant responsibility of protecting...
Partial Processing Recovery: Resuming Failed Pipeline Steps
In the age of big data, analytics pipelines form the cornerstone of informed and agile strategies for companies aiming to innovate faster and optimize...
Backfill Strategies for Historical Data Processing
Historical data processing can feel like digging into an archaeological expedition. Buried beneath layers of data spanning months—or even years—lies...
Continuous Integration for Data Transformation Logic
In the dynamic landscape of data-driven businesses, speed and accuracy are paramount. Organizations increasingly rely on complex data transformation...
Optimistic vs. Pessimistic Locking in Data Integration Processes
In today's interconnected business landscape, data drives decisions, powers innovation, and inspires new opportunities. Effective data integration is...
Pipeline Orchestration: Airflow vs. Prefect vs. Dagster Comparison
In the data-driven world we operate in today, robust and efficient pipeline orchestration is not just a technical luxury—it’s a vital cornerstone of...
Checkpoint-Based Recovery for Long-Running Data Transformations
Imagine running a critical data transformation task that's been processing for hours or even days, only to experience a sudden crash or unexpected system...
Configuration-Driven Pipeline Design vs. Hard-Coded Logic
In today's dynamic technology landscape, organizations must evolve swiftly to leverage data effectively. The decisions we make now regarding data pipeline...
Data Pipeline Branching Patterns for Multiple Consumers
In today's increasingly data-driven market, companies that leverage their information assets effectively achieve a distinct competitive edge. However, as...
Data Transformation Debugging Techniques and Tools
In our increasingly data-driven landscape, transforming raw data into meaningful insights sits at the core of every successful business strategy. Yet, for...
Extract-Load-Transform vs. Extract-Transform-Load Architecture
In an era increasingly driven by data, organizations across every industry stand at a critical crossroads of choosing the right data integration approach....
Long-Running Transaction Management in ETL Workflows
Organizations today thrive on their ability to quickly convert vast and constantly evolving data into actionable insights. ETL (Extract, Transform, Load)...
Parameterized Pipeline Templates for Reusable Data Processing
In an era where speed, efficiency, and scalability define competitive advantage, businesses continuously seek smarter methodologies to handle their data...
Data Pipeline Dependency Resolution and Scheduling
In the fast-paced world of data analytics and innovation, businesses constantly seek strategies to streamline their operations, enhance reliability, and...
Functional Programming Paradigms in Data Transformation Logic
Today's intricate data landscapes demand intelligent approaches to transform raw data into meaningful and actionable insights. As data continues to...
Pipeline-as-Code: Infrastructure Definition for Data Flows
In an increasingly data-driven world, harnessing massive volumes of information requires sophisticated, scalable, and resilient infrastructure....
Transactional Data Loading Patterns for Consistent Target States
Imagine the foundation of your organization's strategic success as a skyscraper built from carefully assembled blocks of accurate data. Each transaction...
Code Generation for High-Performance Data Transformations
In today's fast-paced business environment, decision-makers depend heavily on accurate, timely, and insightful analytics. Behind these insights lies one...
Data Enrichment Pipeline Architecture Patterns
In a rapidly evolving data management landscape, successful organizations are no longer content simply collecting vast amounts of raw data; today's...