In Sparkflows, we can use the ‘Row Filter’ processor to apply filter based on multiple conditions joined using ‘and’ and ‘or’ operators.
To use the ‘Row Filter’ Processor:
- Enter a ‘Conditional Expression’ consisting of multiple conditions joined together. An example would be ‘salary > 20000 and dept_no = 10’. It would filter rows based on the expression and send them to the outgoing DataFrame.
For more information read the Sparkflows Documentation here:
Hi Nagisa,
In Sparkflows, we can use the ‘Row Filter’ processor to apply filter based on multiple conditions joined using ‘and’ and ‘or’ operators.
To use the ‘Row Filter’ Processor:
- Enter a ‘Conditional Expression’ consisting of multiple conditions joined together. An example would be ‘salary > 20000 and dept_no = 10’. It would filter rows based on the expression and send them to the outgoing DataFrame.
For more information read the Sparkflows Documentation here:
https://docs.sparkflows.io/en/latest/user-guide/data-preparation/filter.html?highlight=row%20filter#row-filter