Release Notes v2.7 (November 2022)
Pipelines
Run pipelines without credentials being expired
Long pipelines can now run to completion as credentials no longer expire after 12 hours.
Increased Pipeline Capsule Resource Limits
The CPU and memory resources for Pipeline Capsules have been increased from 52 cores and 100 GB to 192 cores and 4096 GB.
Use of quotes in Pipeline Capsule Arguments
Nextflow Configuration File for Pipelines
Users can now use their own Nextflow configuration file in a pipeline
Release Capsule Version Control for Pipelines
Users can choose which version of a release capsule to use when adding a new capsule to a pipeline.
The version can be changed at any time in the capsule settings.
The version used by the pipeline will remain unchanged if a new version of a capsule is released.
Run Pipelines on On-demand EC2 Instances
Each capsule runs on a dedicated EC2 instance during a reproducible pipeline run. Previously spot instances were used, which caused runs to fail due to the instance becoming unavailable.
Autoscaling EBS for Intermediate and Final Results
With the implementation of autoscaling EBS volumes, pipelines intermediate and final results can now be almost unlimited in size.
Resume Pipeline Run from any Intermediate Step
A failed or successful run can be resumed from an intermediate step instead of restarting from the beginning.
Autocomplete Package Names
The system will have a dropdown list of the package name, depending on the package manager selected.
Note: This list is case sensitive and can also suggest items when searched for as sub-strings.
User Interface Feedback
When a dedicated machine/resource is no longer available, the feedback displays on the screen.
Concurrent Reproducible Run Overrides Run Output
The run output can be switched between concurrent runs in the console by clicking View Output.
Capsule Viewers can Attach Secrets
Anyone with View Access can attach secrets to the capsule that is shared with them.
Open Science Library
Gallery capsules now have a dedicated dashboard that displays curated capsules.
Data Improvement
Creating New Data Flow
With the new UI for creating Data Asset in a Cloud Workstation, all metadata for creating a new dataset is now available on one window.
When the Data Asset name is entered, a folder name is simultaneously generated, and tags can also be entered.
The same options are available for importing Data from S3 and Google Cloud.
The UI for importing from cloud data sources, for example, S3, has changed from Keep files on external storage to Import from an S3 bucket or Link to an S3 bucket.
There is a warning for creating Data Assets that, once created, Data Assets are immutable (unable to be changed) for reproducibility.
For private buckets, credentials no longer need to be specified. The system will automatically go through the user's Roles and Secrets to check for access. If no access is available, users will have the option to provide a new User Secret.
Attaching a New Data Flow
With the new UI for attaching Data in the capsule IDE and the Cloud Workstation, it is easy to toggle between All and Attached Data in one window instead of opening a popup for attaching new Data.
Data can be searched for by name in both attached and unattached Data.
From the Attach/Detach Data page, metadata can be viewed without leaving the Cloud Workstation page. This information can be retrieved by hovering over the dataset, which does not have to be attached and clicking Data Details.
While in the Cloud Workstation, the same Data information will display as it's displayed on the Data dashboard.
List Capsules Using a Dataset
Each Data Asset in Code Ocean has a unique ID that can be viewed in the Data Details window. in either the Data dashboard or via a popup in the capsule IDE.
The Data Asset ID can be copied to the clipboard and used to search for capsules that have the Data Asset attached.
Using the ID in the same manner a capsule name search is conducted, the search takes place in the capsule dashboard.
Updates for Administrators
Admin Guide
Administrators can access the Admin Guide in the Admin Panel under Support.
Analytics for Admin
The Admin Panel now includes an analytics section, where administrators have more visibility of different activities in their environment:
User activity and estimated costs
Compute resources used
Capsule usage information which includes links to the capsules
Storage used
Data - types, permissions, links to capsules
Tags - used in either capsules or Data
Administrators can use Filters to focus on specific time-frames, users, capsules, and Data, among others.
Data Improvement
Assume Roles
Assume Roles allows organizations to leverage AWS Identity and Access Management (IAM) best practices. By using IAM Roles, administrators can securely manage who has access to specific AWS services and resources. End users no longer need to store or define AWS secrets, creating a more simplified user experience.
A Role is a collection of permissions, and a user or group inherits these permissions when they assume the role.
Security best practices use the Least privileges model, meaning users have the minimum amount of access required. Privileges are elevated selectively when needed.
Roles Automation consists of:
Creation of External Data/access to an external S3 bucket
AWS services such as AWS batch and AWS managed databases. Administrators create IAM Roles and set permission policies in AWS.
Administrators allow users or groups to assume the Roles via SAML SSO IdP integration.
Last updated
