Skip to main content

Using Parameters in the Notebook Process

Parameterization is a foundational capability in the Syntasa platform that transforms a static Jupyter Notebook into a dynamic, reusable, and production-ready process. By defining and injecting parameters at runtime, a single notebook can be executed across multiple dates, datasets, environments, or business scenarios—without modifying the underlying source code.

This article provides a comprehensive technical reference for implementing, configuring, and validating parameters in the Notebook Process, with a focus on correctness, maintainability, and production best practices.

The Foundation: The parameters Cell Tag

Syntasa uses the industry-standard Papermill approach for parameter injection. To enable parameterization, explicitly tag a cell (recommended first cell) as a parameters

How to Tag a Cell in JupyterLab​

  1. Open the notebook in a Notebook Workspace.
  2. Select the cell that contains variable definitions, for example:
database= "xyz"
process_date = "1970-01-01"
is_production = false
  1. In the right-hand sidebar of JupyterLab, open the Property Inspector.
  2. Expand the Cell Tags section.
  3. Enter parameters (all lowercase) and click Add.
  4. Save the notebook.

Technical Notes​

  • The Notebook Process scans for the first cell tagged as parameters.
  • Value can be defined to the variables in this cell.
  • These values may be overridden at runtime by values configured in App workflow.

Important: The tag must be exactly parameters (case-sensitive). Any variation will prevent detection.

Synchronizing Parameters with App workflow

After tagging and saving the notebook, App workflow must be updated to recognize the new or modified parameters.

Synchronization Steps​

  1. Open App workflow and select the Notebook Process node.
  2. Verify that the correct Workspace and Notebook Name are selected.
  3. Click Refresh in the configuration sidebar.

What Happens During Refresh​

  • The system re-scans the .ipynb file.
  • The tagged parameters cell is identified.
  • Variable names and default values are extracted.

The detected variables are displayed in the Parameters Preview section of the UI.

Mapping Notebook Parameters to App Variables

The true power of parameterization lies in mapping notebook variables to Syntasa App Variables or System Macros.

Common Mapping Patterns​

Date-Driven Processing

process_date → {{process_date}}

Ensures that each execution receives the correct processing date based on the App’s schedule.

Batch Tracking and Lineage

batch_id → {{batch_id}}

Maintains traceability across pipeline executions.

Environment Flags

is_production → true / false

Allows conditional logic within the notebook to differentiate between development and production behavior.

Supported Data Types​

The Notebook Process supports standard JSON-compatible types:

  • Strings: "2024-10-27"
  • Integers / Floats: 100, 0.05
  • Booleans: true, false

Best Practice: Ensure the injected value type matches the default value type defined in the notebook.

Execution Model: How Parameter Injection Works

When a Notebook Process is triggered—either manually or by a schedule—the platform follows a deterministic injection sequence:

  • Runtime Initialization
    The configured runtime environment (Python or Spark) is launched.
  • Injected Parameters Cell Creation
    App workflow generates a new cell immediately after the tagged parameters cell.
  • Variable Override
    The injected cell assigns the runtime values, for example:
process_date = "2024-10-27"
batch_id = "b_12345"
  1. Notebook Execution
    All subsequent cells execute using the overridden values.

Because the injected cell runs after the defaults, runtime values always take precedence.

Verifying Runtime Values (Executed Notebook)

Every Notebook Process execution produces an Executed Notebook artifact.

How to Verify Parameters​

  1. Navigate to App Monitor after the run completes.
  2. Select the Notebook Process node.
  3. Open the Executed Notebook from the Outputs section.

Within the notebook, an Injected Parameters cell is visible, clearly showing the exact values used for that execution.

This provides a transparent audit trail for validation, debugging, and compliance.

Best Practices and Troubleshooting

Best Practices​

  • Limit Scope: Include only variables that need to change at runtime.
  • Type Consistency: Keep default and injected value types consistent.
  • Top-Level Placement: Place the parameters cell near the top of the notebook (typically after imports) for clarity and maintainability.

Troubleshooting Guide​

Parameters Do Not Appear in App workflow

  • Was the notebook saved in JupyterLab?
  • Is the cell tag exactly parameters (lowercase, no spaces)?
  • Was the Refresh button clicked in App workflow?

Values Are Not Overriding Defaults

  • Check whether the variable is redefined later in the notebook.
  • Ensure logic does not overwrite the injected value.

System Macro Errors

  • Verify the App has a defined schedule or execution context.
  • Ensure system macros (for example, {{process_date}}) are supported by the trigger type.

Summary Checklist

Before running a parameterized Notebook Process, confirm the following:

  • A notebook cell is tagged with parameters.
  • The notebook is saved in JupyterLab.
  • Refresh has been clicked in App workflow.
  • Parameters are mapped to system macros or static values.
  • Runtime values are verified in the Executed Notebook snapshot.

Conclusion

Parameterization is a critical enabler for scalable and maintainable notebook-based pipelines. By following a consistent tagging strategy, synchronizing changes with App workflow, and validating execution artifacts, teams can confidently reuse notebooks across workflows while maintaining transparency and production reliability.