Snowflake Query v2 activity

Introduction

A Snowflake Query v2 activity, using its Snowflake connection, retrieves a CSV file of table or view data across multiple Snowflake schemas and is intended to be used as a source to provide data in an operation.

Create a Snowflake Query activity

An instance of a Snowflake Query v2 activity is created from a Snowflake connection using its Query v2 activity type.

To create an instance of an activity, drag the activity type to the design canvas or copy the activity type and paste it on the design canvas. For details, see Create an activity or tool instance in Component reuse.

An existing Snowflake Query v2 activity can be edited from these locations:

• The design canvas (see Component actions menu in Design canvas).
• The project pane's Components tab (see Component actions menu in Project pane Components tab).

Configure a Snowflake Query activity

Follow these steps to configure a Snowflake Query v2 activity:

• Step 1: Enter a name and select an object
  Provide a name for the activity and select an object.

• Step 2: Build your query
  Set conditions on a query using the object fields and apply paging to a query.

• Step 3: Review the data schemas
  Any request or response schemas generated from the endpoint are displayed.

Step 1: Enter a name and select an object

In this step, provide a name for the activity and select a table or view (see Snowflake's Overview of Views). Each user interface element of this step is described below.

• Endpoint menu: If you have multiple endpoints of the same connector type configured, a menu at the top of the screen displays the current endpoint name. Click the menu to switch to a different endpoint. For more information, see Change the assigned endpoint in Configuration screens.

  • Edit endpoint: Appears when you hover over the current endpoint name. Click to edit the currently selected endpoint's connection configuration.

• Name: Enter a name to identify the activity. The name must be unique for each Snowflake Query v2 activity and must not contain forward slashes / or colons :.

• Select Table(s)/View(s): This section displays tables and views available in the Snowflake endpoint. When reopening an existing activity configuration, only the selected table or view is displayed instead of reloading the entire list.

  • Filter: Enter a filter for the table or view name to load, using % as a SQL wildcard to match any string. For example, enter INFORMATION_SCHEMA.% to list all tables and views in the INFORMATION_SCHEMA. Click Refresh after entering a filter to load the matching results.

  • Refresh: Click the refresh icon or the word Refresh to load or reload tables and views from the Snowflake endpoint using the current filter. A filter must be entered before refreshing. This may also be useful if tables or views have been recently added to Snowflake.

  • Selecting tables or views: After refreshing, matching tables and views are displayed in a list on the left. Click a table or view to add it to the selection table on the right. Selected items are highlighted. Any number of tables and views may be selected to build a multi-table query with joins.

    Tip

    If the list does not populate with available tables or views after entering a filter and refreshing, the Snowflake connection may not be successful. Ensure you are connected by reopening the connection and retesting the credentials.

  • Selected tables and views table: Selected tables and views are displayed in a table on the right. Each column is described below:

    • Table: The name of the selected table or view.

    • Parent: For each child table or view being joined, use the dropdown to select its parent. The first table selected has no parent and serves as the root of the join hierarchy. Once a parent is selected, an Assign link appears in the Link Keys column.

    • Join Type: After link keys have been assigned for a child table or view, use the dropdown to select the relationship type between the child and its parent:

      • One or More: Each parent record must have at least one child record.

      • One Only: Each parent record must have exactly one child record.

      • Zero or More: Each parent record can have zero or more child records.

      • Zero or One: Each parent record can have zero or one child record.

    • Link Keys: For each child table or view with a Parent selected, click Assign to open a separate window where you define the key relationships between the parent and child:

      • Parent object: Displays the parent table or view and its fields on the left. Use the search box to filter fields.

      • Child object: Displays the child table or view and its fields on the right. Use the search box to filter fields.

      • Assign link key(s): Drag a field from the parent to the corresponding field in the child (or vice versa) to create a link. Repeat to assign multiple key pairs.

      • Unassign link key(s): Click a field that has already been linked to remove its link assignment.

      • Start over: Click to clear all assigned link keys.

      • Finish: Click to save the link key assignments and close the window. The number of assigned link keys appears in the Link Keys column and the Join Type dropdown becomes available.

  Note

  The Next button remains disabled until all tables and views with a parent selected have their link keys assigned. A warning is displayed at the bottom of the screen if any link key assignments are outstanding.

• Save & Exit: If enabled, click to save the configuration for this step and close the activity configuration.

• Next: Click to temporarily store the configuration for this step and continue to the next step. The configuration will not be saved until you click the Finished button on the last step.

• Discard Changes: After making changes, click to close the configuration without saving changes made to any step. A message asks you to confirm that you want to discard changes.

Step 2: Build your query

In this step, build a query statement by setting conditions for object fields and applying paging either through the query builder or by manually entering a query statement. Each user interface element of this step is described below.

• Search: Enter any part of a field name into the search box to filter the list of fields for the selected object. The search is not case-sensitive. The listed results are filtered in real time with each keystroke.

• Refresh: Click the refresh icon or the word Refresh to reload fields of the object from the Snowflake endpoint.

• Select All: When using the search box to filter, you can use this checkbox to select all visible fields at once.

• Select fields: Select the checkboxes of the fields you want included in the query to have them automatically added to the query statement in the Query string. You can also Select All of the fields at once using the checkbox.

• Paging: To add a paging clause (a limit on the number of records with an optional record offset), you can use the dropdown to set the paging limit and the field to enter an offset. If an offset is not specified, it defaults to 0. A single paging clause is supported. If paging clause is not included, all records are returned.

  • Apply: Click to automatically construct the clause based on the dropdown selections and entered value. The automatically constructed paging clause appears in the Query string text box.

  • Remove: Click to remove a paging clause that has been applied.

• Conditions: To add conditional clauses, use the fields below as input to help construct the clauses, which then appear in the Query string text box.

  • Field: Use the dropdown to select a field from the selected object.

  • Operator: Use the dropdown to select an operator that is appropriate for the field data type:

    Operator	Label	Description
    =	Equals
    !=	Not equals
    IN (value1, value2)	In	In list of values.
    LIKE 'string'	Like	Like string.
    LIKE 'string%'	Starts with	Starts with string.
    LIKE '%string'	Ends with	Ends with string.
    LIKE '%string%'	Contains	Contains string.
    <	Less than
    <=	Less or equal
    >	Greater than
    >=	Greater or equal

  • Value: Enter the desired value to use with the dropdown selections.

  • Add: Click to automatically construct the clause based on the dropdown selections and entered value. The conditional clause is added to the Query string text box.

  • Remove All: Click to remove all entered conditional clauses.

• Query string: As you select fields, specify conditions, and set paging, the query statement in this text box is autopopulated with the selected fields, conditions, and paging limits. This field is editable, meaning you can manually enter a query statement or edit the autopopulated statement.

  Note

  The values of any global variables used in the Query string are not populated when using the Test Query button, even if a default value is specified. Global variable values will be obtained at runtime when the query is executed. To test the query with a default variable value, use a project variable instead.

  Important

  To use a query statement that contains elements other than those selected via this interface, you must also select the Use Static Schema checkbox.

• Use Static Schema: Select to set the response schema to a static schema that is independent of the query statement. This allows custom query statements entered in the Query string text box to be used.

• Test Query: Click to validate the query. If the query is valid, a maximum of 50 records retrieved from the query is displayed in a table. If the query is not valid, relevant error messages are displayed.

  Note

  During operation runtime, the 50 record limit is not enforced unless it is specified in the Paging field (described earlier). If you edit the Query string manually, the query must be valid and validated through this button in order to enable the Next button.

• Back: Click to temporarily store the configuration for this step and return to the previous step.

• Next: Click to temporarily store the configuration for this step and continue to the next step. The configuration will not be saved until you click the Finished button on the last step.

• Discard Changes: After making changes, click to close the configuration without saving changes made to any step. A message asks you to confirm that you want to discard changes.

Step 3: Review the data schemas

Any request or response schemas generated from the endpoint are displayed. Each user interface element of this step is described below.

• Data Schemas: These data schemas are inherited by adjacent transformations and are displayed again during transformation mapping.

  Note

  Data supplied in a transformation takes precedence over the activity configuration.

  The default request and response data schemas consist of these nodes and fields:

  • Request

    Request Schema Field/Node	Notes
    queryRequest	Node of the query request.
    whereClause	Node of the WHERE clause request.
    columnName	Column name of the WHERE clause.
    condition	Condition of the WHERE clause.
    value	Value of the WHERE clause.

  • Response

    Response Schema Field/Node	Notes
    table	Node showing the table name NAME.
    Entity	Node of the entity.
    column_A	Value for the first column, COLOR.
    column_B	Value for the second column, FLOWER.
    . . .	Values for the succeeding table columns.

  When Use Static Schemas is selected, the response data schema consists of these nodes and fields:

  • Response

    Response Schema Field/Node	Notes
    queryResponse	Node of the query response.
    row	Node of the response row.
    item	Node of the response item.
    columnName	Column name of the response item.
    columnValue	Column value of the response item.

  Refer to the Snowflake SQL commands and Snowflake API reference for additional information.

• Refresh: Click the refresh icon or the word Refresh to regenerate schemas from the Snowflake endpoint. This action also regenerates a schema in other locations throughout the project where the same schema is referenced, such as in an adjacent transformation.

• Back: Click to temporarily store the configuration for this step and return to the previous step.

• Finished: Click to save the configuration for all steps and close the activity configuration.

• Discard Changes: After making changes, click to close the configuration without saving changes made to any step. A message asks you to confirm that you want to discard changes.

Next steps

After configuring a Snowflake Query v2 activity, complete the configuration of the operation by adding and configuring other activities or tools as operation steps. You can also configure the operation settings, which include the ability to chain operations together that are in the same or different workflows.

Menu actions for an activity are accessible from the project pane and the design canvas. For details, see Activity actions menu in Connector basics.

Snowflake Query v2 activities can be used as a source with these operation patterns:

• Transformation pattern
• Two-target archive pattern (as the first source only)
• Two-target HTTP archive pattern (as the first source only)
• Two-transformation pattern (as the first or second source)

To use the activity with scripting functions, write the data to a temporary location and then use that temporary location in the scripting function.

When ready, deploy and run the operation and validate behavior by checking the operation logs.