Create a pipeline in 5 minutes (Update)

This demonstration provides the steps for building a simple pipeline and processing data in parallel using Code Ocean Pipelines.

1. Create an example dataset containing reads from different samples. All reads for each sample will be in a separate folder. The pipeline will pass each folder to the downstream alignment, allowing each sample to process in parallel.

   • Bucket Name - codeocean-public-data

   • Path - example_datasets/Normox

2. Click My Pipelines to navigate to Pipelines

3. Click New Properties, Create New to create the pipeline

4. Click Manage Data Assets

5. Search for the Data Asset - Normox

6. Drag and drop the data asset onto the grid, to add it to the interface

7. Click to expand the menu

   • My Capsules are from the Repository

   • Code Ocean Apps are prebuilt capsules from Code Ocean

8. Add FastQC capsules to the pipeline

9. Select the latest version

10. Connect Reads to FastQC

11. Connect FastQC to Results Bucket

12. Click
13. Select Default as the Connection type

14. Click Confirm

15. Click Commit Changes in Reproducible Run

To change where the output files are stored adjust the Destination parameter

15. Click Reproducible Run

Normox had 6 samples with sequencing files

Instances of FastQC are spun up to process these samples in parallel

Once the run is complete, the results can be downloaded locally or included into a Data Asset

Pipeline with Parameters

The previous implementation used default parameters of the capsule. This demonstration provides parameters via App Builder for pipelines.

1. Click App Builder, Create App

2. Provide a subtitle for the pipeline

3. Include instructions explaining the function of the use cases of the pipeline in App instructions

4. Click Add to add data

5. Complete the fields, see App Panel Builder for details

6. Click Save

Under Parameters capsules can be arranged into any order and will be displayed on the App Panel in this order.

7. Click Finish

8. Click Run

The number of threads as per the FastQC instance, logging and levels, Normox has six samples with sequencing files. There are six instances of FastQC are spun up to process these samples in parallel

9. Once the run is complete, results can be downloaded locally or put into a data asset