Sparkflows provides the Flatten & Explode nodes to extract the columns from complex data types like struct type & array type.
Flatten: Creates the single columns from struct.
Explode: Explode an array columns to rows.
Nodes in fire:
NodeFlatten and NodeExplode
Example Data
Schema
Explode the Topping Column
Explode the array of topping column into rows with new column as explode_topping
Flatten the batters and explode_topping
Final Output
After dropping the few columns
Comments