Create a pipeline in 5 minutes
Before creating the pipeline, convert all local data files into Data Assets as the same data will be used in the pipeline. Using data assets will improve the performance of the pipeline by ensuring data is passed smoothly through Nextflow channels. To learn more about data assets refer to the Data Assets Guide.
Create a data asset from sample-data.txt with the following steps:
Download the sample-data.txt file to your computer
Click Manage Datasets at the data folder in your file tree system
Click Attach Data
Click Add Dataset
Select Local files
Click Next
Click Choose Files
Select sample-data.txt file
Name the dataset and default folder Sample_Data and a tag of test.
Click Add dataset
Create a new Pipeline
To navigate to the Pipeline:
Click Pipeline
Click Add Pipeline, Create New
Add the Capsule to the Pipeline
In the Add Capsules menu, find "My First Capsule" and drag it onto the pipeline building area.
Add a Code Ocean Apps Capsule
Pre-built capsules can be leveraged from the Code Ocean Apps Library in Pipelines by dragging and dropping them into the Pipeline Builder UI.
Form a Capsule Connection
To ensure that the capsule's results are saved after each run, connect the capsule to the results bucket:
Click + at the bottom of the capsule
Attach the Data Asset
To attach the Sample_Data data asset to the pipeline's data folder:
Click Manage Datasets at the data folder of the file tree system
Click Attach Data
Select the Sample_Data data asset from the dropdown menu
Passing the file to the Destination Capsule
In the current configuration of the pipeline, the capsule expects to find a sample-data.txt file in the data folder but finds a folder named Sample_Data. To correct this:
Click on the gear in the middle of the data to capsule connection
In the pop-up menu set the source path to data/Sample_data/sample-data.txt
3. To run the pipeline click Reproducible Run at the top right of the pipeline page
4. Once the run is complete, the contents of the result bucket can be viewed in the Timeline
This tutorial demonstrated how to create a simple single-capsule pipeline but the Visual Pipeline Builder also supports the creation of complex multi-capsule pipelines.
Viewing the Pipeline
There are four options for viewing the pipeline:
Zoom in - magnifies the pipeline
Zoom out - minimizes the pipeline
Fit view - shows the pipeline in its entirety
Eye toggle - shows the:
Connection types
CPUs
Memory
Data Type: Internal, External and Result
Capsule Type: Release, Code Ocean Apps
Export A Pipeline
Pipelines can be exported from the Pipeline menu.
Exporting the pipeline automatically creates nextflow.config
and REPRODUCING.md
files which contain all the configurations and instructions required to run the pipeline on a local machine.
Export a Pipeline
Last updated