Publishing & Versioning
Last updated
Was this helpful?
Last updated
Was this helpful?
Imagine your data stream requires modifications and you do not wish to redo your Pipeline configuration every time. Onum allows you to simultaneously work on multiple versions of the same Pipeline.
When you modify a Pipeline, you will be creating a version history, which appears in the top right.
When you create or modify a Pipeline for the first time, you will create a Draft. This will be where you can build your Pipeline before deciding to publish it. This means you can work on various versions in the background, whilst choosing only one to be published.
Once you are happy with the Draft version of your Pipeline, there are various things you can do.
Before moving further, you can test your Pipeline to see how the results will look without publishing it. Click Test Pipeline, enter the maximum number of events to allow and select Debug.
Click through the different tabs to see your data as it moves through each Pipeline element. You can also view the specifics of what enters and exits each port.
If you have finished your Pipeline and would like to confirm its completion, you can click the Publish button in the top right. Publishing is a way to validate this version.
Now, you can decide whether to run this Pipeline on a Cluster (see the section below) or simply validate this version without pushing it to the cluster.
After clicking Publish, you can decide to run this Pipeline on selected or all clusters.
Select the desired clusters and click Publish.
Choosing to Cancel this action will stop the Pipeline from running on the cluster, but will not undo the Publish action.
Notice the Running In menu that appears. When you open a Pipeline, the default version that opens will be Running.
Use the Running in Clusters drop-down menu to view a list of the clusters your Pipeline is currently running in.
Update these settings by selecting the Cluster from the list and clicking Update x Clusters.
You can view running and run Pipelines from the configuration pane, as well as select which clusters to do so in.
The cluster selector only appears when you have more than 10 clusters to choose from.
You can Stop a Pipeline that is running in a Cluster at any time using the Stop Pipeline button that appears on any running Pipeline.
Select which, if not all, clusters to stop running on and confirm Stop Pipeline again.
When stopping a Pipeline, the worry is that your data will be lost. Don't worry! This will simply save your Pipeline in the Stopped versions list.
You can change the description using the pencil icon.
In the Pipeline Settings, you can see a comprehensive list of all of its previous versions and the changes in each.
Choosing to Publish a Pipeline will generate a new version, which you can view in the tree, or in the Pipeline preview. Expand one to see its details and click the pencil icon to edit the name.
Use the Running in drop-down to see which clusters it is running in.
Aside from the Pipeline version, each Listener, Action and Datasink has its own version to ensure they are constantly evolving.
You can see the current version of each in the article dedicated to it.
For older versions, go to the section.
