Node Overview
Nodes provide template structure for performing data extraction, processing, and API communication. The diagram below shows how a node performs a base function, but exposes an interface for user-defined code to enable tailoring.

In Jupyter notebooks associated backing editable nodes, user-defined SQL and user-defined Python cells share a light blue background and are committed to a repo + deployed to a workflow orchestrator upon running the Save Pipeline Code cell at the bottom of the notebook, or by pressing Cmd+Shift+S
or Ctrl+Shift+S
(Windows/Linux).
Key Node Characteristics
A useful way to interact with nodes is to consider how nodes interact with other systems, their input and output types, and whether or not they contain a user-editable component. The table below shows the full list of available nodes, along with whether there is a user-editable component associated with the node.
Available Nodes with Key Characteristics
The table below lists available nodes.
- Is Editable indicates that the node has a user-editable components
- Is Multi indicates that the node can accept multiple inputs.
Category | Name | Input Types | Output Types | Is Editable | Is Multi |
---|---|---|---|---|---|
Analysis | Branch_Python | Table(s) and/or File(s) | NodeReturn | True | False |
Analysis | Python | Table(s) and/or File(s) | NodeReturn | True | False |
Analysis | RunContainer | FileAny | FileAny | True | False |
Analysis | Transform_SQL | Table | Table | True | False |
Analysis | Transform_py | Table or List[Table] | NodeReturn | True | False |
Analysis | Trigger_Python | Table(s) and/or File(s) | FileAny | True | False |
App | APINode | API | NodeReturn | True | False |
App | AirtableExport | Table | API | True | False |
App | AirtableImport | API | NodeReturn | False | False |
App | Azure_Query | API | NodeReturn | True | False |
App | Azure_Read | API | FileAny | False | False |
App | Azure_Read_Multi | API | FileAny | False | True |
App | Azure_Write | FileAny | API | False | False |
App | Benchling_Api | API | NodeReturn | True | False |
App | Benchling_Event | App | FileAny | True | False |
App | Benchling_Read | API | NodeReturn | True | False |
App | Benchling_Read_Object | API | NodeReturn | True | False |
App | Benchling_Write | Table | NodeReturn | True | False |
App | Benchling_Write_Object | Optional[FileAny or List[FileAny]] and Optional[Table or List[Table]] | NodeReturn | True | False |
App | Coda_Write | Table | NodeReturn | True | False |
App | ELabNext_Write | Table | NodeReturn | True | False |
App | Load_Parquet_to_Table | API | FileAny | False | False |
App | S3_Event | App | FileAny | True | False |
App | S3_Read | API | FileAny | False | False |
App | S3_Write | FileAny | API | False | False |
App | SciNote_API | Table | NodeReturn | True | False |
App | Smartsheet_Read | API | NodeReturn | True | False |
App | Snowflake_Write | Table | NodeReturn | False | False |
File | AVI_Read | FileAVI | NodeReturn | True | False |
File | AVI_Read_Multi | Set[FileAVI] | NodeReturn | True | True |
File | CSV_Read | FileCSV | NodeReturn | True | False |
File | CSV_Read_Multi | Set[FileCSV] | NodeReturn | True | True |
File | CSV_Write | Table or List[Table] | NodeReturn | True | False |
File | Excel_Read | FileExcel | NodeReturn | True | False |
File | Excel_Read_Multi | Set[FileExcel] | NodeReturn | True | True |
File | Excel_Write | Table or List[Table] | NodeReturn | True | False |
File | FCS_Extract_Load | FileFCS | NodeReturn | True | False |
File | HDF5_Read | FileHDF5 | NodeReturn | True | False |
File | Image_Read | FileImage | NodeReturn | True | False |
File | Image_Read_Multi | List[FileImage] | NodeReturn | True | True |
File | Image_Write | Table or List[Table] | NodeReturn | True | False |
File | Input_File | FileAny | NodeReturn | True | False |
File | Input_File_Multi | Set[FileAny] | NodeReturn | True | True |
File | PDF_Read | FilePDF | NodeReturn | True | False |
File | PDF_Read_Multi | Set[FilePDF] | NodeReturn | True | True |
File | Powerpoint_Write | Table or List[Table] | NodeReturn | True | False |
File | XML_Read | FileXML | NodeReturn | True | False |
File | Zip_Read | FileZip | NodeReturn | True | False |
Instrument | Agilent_HPLC_Read | File | NodeReturn | True | False |
Instrument | Instron_Tensile_Read | FileIsTens | NodeReturn | True | False |
Instrument | Profilometer_Read | FileHDF5 | NodeReturn | True | False |
Instrument | Synergy_Read | FileTxt | NodeReturn | True | False |
Instrument | Synergy_Read_Multi | List[FileTxt] | NodeReturn | True | True |
Tag | Benchling_Tag | TagBenchling | string | False | False |
Tag | Input_Param | string | string | False | False |
Node Categories
- App: Accesses third-party APIs for processing; in many cases, key exchange between third-party and Ganymede are necessary for functionality
- Analysis: Performs Python / SQL manipulations
- Instrument: Lab instrument-specific functions
- File: For ETL operations on data of specified type into Ganymede cloud
- Tag: For specifying parameters at flow runtime
Node Input/Output Types
Input/output types are split into the following categories:
- NodeReturn: Object holding tables and files to return from Node. Initializing a NodeReturn object involves passing the following parameters:
- tables_to_upload: Dictionary of Pandas DataFrames keyed by table name to store in Ganymede
- files_to_upload: Dictionary of files (bytes objects) keyed by file name to store in Ganymede
Running the following in a cell in the editor notebook
import pandas as pd
def execute():
message = "Message to store in file"
byte_message = bytes(message, "utf-8")
df = pd.DataFrame({"col1": [1, 2, 3], "col2": [4, 5, 6]})
# upload a table named 'my_table' and a file named 'my_file'
return NodeReturn(files_to_upload={"my_file": message}, tables_to_upload={"my_table": df})
execute()
returns the following summary of the NodeReturn object:

Docstrings and source code can be viewed by typing ?NodeReturn
and ??NodeReturn
respectively in a cell in the editor notebook.
- Table: Tabular data retrieved from or passed to tenant-specific Ganymede data lake. Tables are retrieved from Ganymede data lake via ANSI SQL queries, and are passed to Ganymede data lake as pandas DataFrames
- API: access via third-party API
- File-related inputs/outputs: File of specified type.
- FileAVI: AVI file
- FileCSV: CSV file
- FileExcel: Excel file (xls, xlsx, ..)
- FileImage: Image file (png, bmp, ..)
- FileHDF5: HDF5 file
- FileXML: XML file
- FileZip: Zip file
- FileAny: generic data file, which may be unstructured
- TagBenchling: Benchling run tag
- string: String parameter
Set, List, and Dict correspond to Python sets, lists, and dictionaries respectively.
Optional indicates that the input or output is optional.
User-editable Nodes
User-editable nodes present an interface for modifying and testing code that is executed by the workflow management system. These Jupyter notebooks are split into the following sections:
- Node Description: A short blurb about the node that the user-editable function corresponds to
- Node Input Data: For nodes that retrieve tabular data from the data lake as input, the query string in this cell specifies the query (-ies) that are executed and presented to the user-defined function for processing.
- User-Defined Function: The execute function within this cell processes data. The workflow management system calls the execute function within this cell during flow execution.
The execute function may call classes and functions found within the User-Defined Function cell.
-
Save Pipeline Code: This cell stores any changes to the Node Input Data (if present) and User-Defined Function cells.
-
Testing Section: The cells in this section can be used for testing modifications to the SQL query and user-defined python function. This enables rapid iteration on user-defined code; after necessary edits are made, changes can be saved in by running the Save Pipeline Code cell.
List of Available Nodes
Category | Name | Brief Description |
---|---|---|
Analysis | Branch_Python | Process data with Python and conditionally execute downstream nodes |
Analysis | Python | Process data with python |
Analysis | RunContainer | Run container node |
Analysis | Transform_SQL | SQL analysis Function |
Analysis | Transform_py | Manipulate data with python |
Analysis | Trigger_Python | Process data with Python and trigger subsequent flow |
App | APINode | Generic API Access Node |
App | AirtableExport | Export data from Ganymede data lake to Airtable |
App | AirtableImport | Import data from Airtable into Ganymede data lake |
App | Azure_Query | Query data from Azure SQL Server |
App | Azure_Read | Read data from Azure Blob Storage |
App | Azure_Read_Multi | Read all data from Azure Blob Storage |
App | Azure_Write | Write data to Azure Blob storage |
App | Benchling_Api | Read Benchling data into data lake |
App | Benchling_Event | Capture events from Benchling for triggering flows |
App | Benchling_Read | Read Benchling data into data lake using run tag |
App | Benchling_Read_Object | Read Benchling data into data lake using object ID |
App | Benchling_Write | Write to Benchling |
App | Benchling_Write_Object | Write object to Benchling |
App | Coda_Write | Write Coda tables |
App | ELabNext_Write | Create and write eLabNext entry |
App | Load_Parquet_to_Table | Create datalake table from parquet files |
App | S3_Event | Capture events from AWS S3 for triggering flows |
App | S3_Read | Ingest data into Ganymede data storage from AWS S3 storage |
App | S3_Write | Write data to an S3 bucket |
App | SciNote_API | Create and write SciNote entry |
App | Smartsheet_Read | Read sheet from Smartsheet |
App | Snowflake_Write | Sync tables in Ganymede data lake to Snowflake |
File | AVI_Read | Read in contents of an AVI file to a table |
File | AVI_Read_Multi | Read in contents of multiple avi files to a table |
File | CSV_Read | Read in contents of a CSV file |
File | CSV_Read_Multi | Read in contents of multiple CSV files |
File | CSV_Write | Write table to CSV file |
File | Excel_Read | Read Excel spreadsheet |
File | Excel_Read_Multi | Read Excel spreadsheets |
File | Excel_Write | Write Excel spreadsheet |
File | FCS_Extract_Load | Load FCS file to data lake |
File | HDF5_Read | Read HDF5 data |
File | Image_Read | Process image data; store processed images to data store |
File | Image_Read_Multi | Process image data for multiple images; store processed images to data store |
File | Image_Write | Process tabular data; write an image to data lake |
File | Input_File | Read data file and process in Ganymede |
File | Input_File_Multi | Read data files and process in Ganymede |
File | PDF_Read | Read in contents of an PDF file to a table |
File | PDF_Read_Multi | Read in contents of multiple pdf files to a table |
File | Powerpoint_Write | Process tabular data; write a powerpoint presentation to data lake |
File | XML_Read | Read XML file into data lake |
File | Zip_Read | Extract Zip file |
Instrument | Agilent_HPLC_Read | Read an Agilent HPLC file of type .uv, .ms, .ch, or .bin |
Instrument | Instron_Tensile_Read | Load .is_tens file to data lake |
Instrument | Profilometer_Read | Read Mx Profiler data file |
Instrument | Synergy_Read | Load Synergy text file to data lake |
Instrument | Synergy_Read_Multi | Load multiple Synergy texts file to data lake |
Tag | Benchling_Tag | Read benchling tag |
Tag | Input_Param | Input parameter into Flow |