Release Notes 2.19 (April 2024)

Data Connectors - AWS Athena and Databricks

When creating a new Internal Data Asset, users now have the ability to connect directly to AWS Athena or a Databricks SQL Warehouse as a data source. Users can provide a query, the results of which will become the contents of the new Internal Data Asset.

Connecting to Databricks requires creating a Databricks secret from the Account page. When connecting to Athena, you will be able to choose from any AWS secrets or assumable roles associated with your user.

New Data Asset Type - Combined Data

This new Data type, which can be selected from the "New Data" menu, allows you to combine multiple External Data into a new Combined Data, which can then be attached to a Pipeline. Using these new Combined Data, you can now parallelize and parameterize Pipeline data sources at the level of the Data instead of at the Data constituent item level. Additionally, this functionality allows you to easily attach hundreds or thousands of Data to a Pipeline at once without the need to manually move or combine data, avoiding multiple drag-and-drop operations in the UI.

In the first phase of this functionality, it's only possible to create Combined Data from External Data and these Combined Data can only be attached to Pipelines. The ability to combine Internal Data Assets and attach Combined Data Assets to Capsules will be available in a future Code Ocean version.

Public API - Python SDK

The Code Ocean Python SDK provides convenient access to the Code Ocean API from applications written in the Python language. This SDK enables you to manage Code Ocean resources such as Capsules, Data Assets, and Computations in your Python applications. All Python examples in the Code Ocean API Documentation have also now been updated to reference the SDK.

To learn more, visit the Python SDK page of our API guide here.