Sean's Data Process Tips

Embedding data in workflow

2012-01-31T06:47:00.000-08:00

Source: QnA in Knime forum

Contributor ID: jfalgout

Goal
To embed data in workflow, so it can be distribute with the workflow package zip file.

Strategy

Create a directory with files underneath the directory containing the node artifacts for a workflow, the "knime.node" flow variable will be populated the next time you edit the node.

The folder you create within the node's folder has to be named "drop". Otherwise when you save the workspace whatever folders/files you add get deleted.

Test

Not yet

Example to create new Node

2012-01-15T07:57:00.000-08:00

Example to create new Node: http://tech.knime.org/developer/example

Typical workflow for test on specific Culumn

2012-01-09T19:43:00.000-08:00

Source: QnA in Knime forum

Contributor ID: James Davidson

Goal
Derive a new column from a test on existing column

Strategy
1. Use [Row Filter] to do the test and split (with Include & exclude) the original table
2. Use [Java snippet] to generate the value for the new column
3. Use [Concatenate] to merge table back

Comment
1. In some case, [Joiner] my be a shortcut for the goal

2. Do not known whether it is the best solution, since I am just a beginner

Resource
Download James's Package

Remove empty columns in Knime

2012-01-06T18:36:00.000-08:00

Goal
Delete the columns comprise of blank value (0, NaN, NA, NULL or fix string)

Strategy
1. If the target is number column (double or int), use [Low Variance Filter] node
2. Use [Transpose] + [Row Filter] + [Transpose] , suggested by
http://tech.knime.org/forum/knime-general/removing-columns-where-every-value-is-empty
3. Use code snippet in scripts: R, JPyhton, and Java to deal the table

Comment
None of the three works for me.
1. My target columns are string type. [String to Number] node need to be wired to source column manually.
It does not make sense if I have tons of empty columns.
2. Missing Value in [Row filter]'s setting only works on specific column, which implies it is not an automation.
3. I can write R, Python or script outside Knime to do filtering.

Final solution
1. Pre-process the CSV file outside Knime
2. Manually skip the column in [File Reader] setting.