Azure Synapse Deployment - Package Based Approach

Objective

We have previously specified how Azure Synapse Artifacts deployment can be achieved using FlexDeploy in this article. In the current article we are going to elaborate more on how to implement package-based deployment for the same.

Design

There are two approaches we follow here:

Approach A:

First Phase - Configure a Generic Package based project. This is the build only project. Select the JSON objects (say Linked Services, Pipeline etc.), generate the ARM template based on the selected resources and store it in the Git repository. This repository may contain all the ARM templates corresponding to different packages. If you are using a feature-specific Git branching strategy, the ARM template can be maintained within the corresponding feature branches and leveraged during pull request (PR) reviews. Second Phase - Configure a Generic Package based project. This is the actual build-deploy project. Select the ARM template from the Git repository where it was stored in previous phase. Build and store it as FlexDeploy artifact. Initiate the Synapse Release pipeline to deploy the artifacts to target workspaces.

Approach B:

Configure a Generic Package based project. This is the actual build-deploy project. Select the JSON objects (say Linked Services, Pipeline etc.) from Git. Build ARM template and store it as FlexDeploy artifact. Initiate the Synapse Release pipeline to deploy the artifacts to target workspaces. Since all developers’ work is currently being persisted to the same Git branch that is mapped to the Synapse workspace, it is advisable to create individual branches (similar to feature branches) and maintain specific JSON files there. This approach will simplify pull request (PR) handling and improve overall collaboration. Note: To manage dynamic, environment-specific properties, it is advisable to define the replacement properties directly within the JSON files using the $\{\{\}\} syntax. Users can opt for any of the above strategies, based on their specific scenario and overall deployment process.

Flow (Approach A)

Workflow configuration for generating ARM template and upload to Repository

Build Workflow:

Note: For uploading the ARM template to repository, we have created another DevOps repository as az-synapse-arm and configured it in FlexDeploy-Integration with the code SYNAPSEARM.If you have some other repository present in GITHUB or Azure DevOps anywhere and want to use that, then please create accordingly and use the correct Instance Code in the workflow pasted below.

GenerateARMAndSyncToGIT

description: GenerateARMAndSyncToGIT
steps:
  - id: '1'
    name: execute
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: execute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: "# Source directory (parent directory A)\r\nSRC_DIR=$FD_TEMP_DIR\r\n\r\n# Target directory (parent directory B)\r\nDEST_DIR=$FD_TEMP_DIR/JSON\r\n\r\nmkdir -p $FD_TEMP_DIR/JSON\r\nchmod -R 777 $FD_TEMP_DIR/JSON\r\n\r\n# Copy .json files while preserving directory structure\r\nfind \"$SRC_DIR\" -type f -name \"*.json\" | while read -r file; do\r\n    # Get the directory path relative to the source\r\n    rel_path=$(dirname \"${file#$SRC_DIR/}\")\r\n    \r\n    # Create the same directory structure in the target\r\n    mkdir -p \"$DEST_DIR/$rel_path\"\r\n    \r\n    # Copy the file to the target directory\r\n    cp \"$file\" \"$DEST_DIR/$rel_path\"\r\ndone\r\n\r\necho \"All JSON files have been copied successfully.\""
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_LOCK
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '2'
    name: publishArtifacts
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: publishArtifacts
      inputs:
        - name: FDAZ_DEVOPS_INP_FEED
          value:
            value: FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: (FD_PACKAGE_NAME + "-" + ARTIFACTS_PACKAGE_NAME).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_VERSION
          value:
            value: PACKAGE_VERSION
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_DESCRIPTION
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_PATH
          value:
            value: FD_TEMP_DIR + "/JSON"
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
      userInputs: []
      userOutputs: []
  - id: '3'
    name: buildPipeline
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: buildPipeline
      inputs:
        - name: FDAZ_DEVOPS_INP_DEFINITION_NAME
          value:
            value: BUILD_DEFINITION_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_BUILD_DEFINITION_ID
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_QUEUE_ID
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_COMMIT_ID
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_BRANCH_NAME
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_VARIABLES_LIST
          value:
            value: >-
              "workspaceName:cicd-demo-test##artifactsPackageName:" +
              (FD_PACKAGE_NAME + "-" + ARTIFACTS_PACKAGE_NAME).toLowerCase() +
              "##feedName:" + PROJECT_NAME + "/" + FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: (FD_PACKAGE_NAME + "-" + PACKAGE_NAME).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
        - output: FDAZ_DEVOPS_OUT_BUILD_PIPELINE_WEB_URL
      userInputs: []
      userOutputs: []
  - id: '4'
    name: downloadArtifacts
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: downloadArtifacts
      inputs:
        - name: FDAZ_DEVOPS_INP_FEED
          value:
            value: FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: (FD_PACKAGE_NAME + "-" + PACKAGE_NAME).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_VERSION
          value:
            value: PACKAGE_VERSION
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_PATH
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: true
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
      userInputs: []
      userOutputs: []
  - id: '5'
    name: clone
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonGITPlugin
      pluginOperation: clone
      inputs:
        - name: FDGIT_INP_INSTANCE_CODE
          value:
            value: SYNAPSEARM
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_DEST_SUBFOLDER
          value:
            value: ARMTemplates
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_BRANCH
          value:
            value: importedartifacts
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_TREELESS_CLONE
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_DEPTH
          value:
            value: ''
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_SPARSE_CHECKOUT_FOLDERS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '6'
    name: copy
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonFilePlugin
      pluginOperation: copy
      inputs:
        - name: FDFILE_INP_FILE_FILTER
          value:
            value: Template*.json
            isExpression: false
          isEncrypted: false
        - name: FDFILE_INP_FILE_FILTER_EXCLUDED
          value:
            isExpression: false
          isEncrypted: false
        - name: FDFILE_INP_SOURCE_PATH
          value:
            value: FD_ARTIFACTS_DIR
            isExpression: true
          isEncrypted: false
        - name: FDFILE_INP_TARGET_PATH
          value:
            value: FD_TEMP_DIR+"/ARMTemplates/"+FD_PACKAGE_NAME
            isExpression: true
          isEncrypted: false
        - name: FDFILE_INP_CLEAN_DIRECTORY
          value:
            value: 'true'
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '7'
    name: add
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonGITPlugin
      pluginOperation: add
      inputs:
        - name: FDGIT_INP_INSTANCE_CODE
          value:
            value: SYNAPSEARM
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_DEST_SUBFOLDER
          value:
            value: ARMTemplates
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_FILELIST
          value:
            value: .
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '8'
    name: commit
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonGITPlugin
      pluginOperation: commit
      inputs:
        - name: FDGIT_INP_INSTANCE_CODE
          value:
            value: SYNAPSEARM
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_DEST_SUBFOLDER
          value:
            value: ARMTemplates
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_NAME
          value:
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_EMAIL
          value:
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_REVISION_MESSAGE
          value:
            value: '"ARM Template updated: "+ FD_PROJECT_VERSION'
            isExpression: true
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '9'
    name: push
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonGITPlugin
      pluginOperation: push
      inputs:
        - name: FDGIT_INP_INSTANCE_CODE
          value:
            value: SYNAPSEARM
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_DEST_SUBFOLDER
          value:
            value: ARMTemplates
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_BRANCH
          value:
            value: importedartifacts
            isExpression: false
          isEncrypted: false
        - name: FDGIT_INP_FOLLOW_TAGS
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []

Project Configuration

Create a Project by choosing the Classification: Package-based and Project Types as Generic

Once it is created the Source Control needs to be configured, so that we can discover the files from the respective repository.

For example, this is how the source control looks like. Note: In the Advanced tab I haven’t selected ProjectName in the checkout folder script.

If you want to keep the ProjectName in the checkout folder script, then you would require tweaking the shell scripting accordingly for the first step.

Discover the files using File Catalog tab

Now we can create Package upon selecting specific JSON files.

Click on the Configuration tab, select General and update the Build workflow with respective Target group.

We are not configuring any Deploy workflow in this Project for the purpose of this Tutorial, as all we need is to generate the ARM template and upload it to GIT.

Project properties

Follow this link to know about the Azure DevOps Build Pipeline configuration. This must be configured prior to configuring the FlexDeploy workflows and project properties.

Execution

Initiate Build on the selected Package.

Note: The ARM templates are published to Azure DevOps Feed as Artifacts with the package name as a keyword in it. Example: <packagename>-<fdprojectname>-json & <packagename>-<fdprojectname>-arm. Now Microsoft has certain restrictions on the naming of the Azure Artifacts, hence please follow the guide before you name the package that you create in FD or else adjust your workflow to be in compliance. https://learn.microsoft.com/en-us/azure/devops/organizations/settings/naming-restrictions?view=azure-devops

The ARM template is now uploaded to repository

The folder name is following the naming we selected for the package name in FD. If any parameter is to be added/updated or some other manipulation is to be done in the ARM template json in terms of variables or parameters, it can be done. But the integrity of the ARM template should remain intact.

Workflow configuration for fetching the ARM template from the repository and initiate the Synapse Release pipeline

Build Workflow:

SynapsePackageBuildARM

description: SynapsePackageBuildARM
steps:
  - id: '1'
    name: Execute
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: execute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: "# Define the name of the zip file to be created\r\nZIP_NAME=$FD_PROJECT_NAME.zip\r\n\r\nmkdir -p $FD_TEMP_DIR/ARMTEMPLATES\r\nfind $FD_TEMP_DIR -type f -name \"Template*.json\" -exec cp {} $FD_TEMP_DIR/ARMTEMPLATES \\;\r\ncd $FD_TEMP_DIR/ARMTEMPLATES\r\n\r\n# Find and zip all Template*.json files into a single zip file\r\nzip \"$ZIP_NAME\" *\r\ncp $FD_TEMP_DIR/ARMTEMPLATES/$ZIP_NAME $FD_ARTIFACTS_DIR\r\n\r\necho \"Zip created and moved to artifacts.\"\r\n"
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_LOCK
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: true
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []

Deploy Workflow:

SynapsePackageDeployARM

name: SynapsePackageDeployARM
description: SynapsePackageDeployARM
variables:
  - code: ENV_DETAILS
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: RELEASE_ID
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: ENVIRONMENT_ID
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: RELEASE_URL
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: PKG_NAME
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: FINALVERSION
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
steps:
  - id: '1'
    name: execute
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: execute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: "mkdir -p $FD_TEMP_DIR/ARMFiles\r\nunzip $FD_ARTIFACTS_DIR/*.zip -d $FD_TEMP_DIR/ARMFiles"
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_LOCK
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: true
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '2'
    name: propertyReplacement
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonFilePlugin
      pluginOperation: propertyReplacement
      inputs:
        - name: FDFILE_INP_SOURCE_PATH
          value:
            value: FD_TEMP_DIR + "/ARMFiles"
            isExpression: true
          isEncrypted: false
        - name: FDFILE_INP_FILE_FILTER
          value:
            isExpression: false
          isEncrypted: false
        - name: FDFILE_INP_FILE_FILTER_EXCLUDED
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '3'
    name: Assign
    type: ASSIGN
    data:
      from: >-
        PACKAGE_VERSION.substring(0,PACKAGE_VERSION.lastIndexOf(".") +1) +
        FD_WORKFLOW_EXECUTION_ID
      to: FINALVERSION
  - id: '4'
    name: publishArtifacts
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: publishArtifacts
      inputs:
        - name: FDAZ_DEVOPS_INP_FEED
          value:
            value: FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: >-
              (FD_PACKAGE_NAME + "-" + PACKAGE_NAME + "-" +
              FD_ENVIRONMENT_CODE).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_VERSION
          value:
            value: FINALVERSION
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_DESCRIPTION
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_PATH
          value:
            value: FD_TEMP_DIR + "/ARMFiles"
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
      userInputs: []
      userOutputs: []
  - id: '5'
    name: createRelease
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: createRelease
      inputs:
        - name: FDAZ_DEVOPS_INP_RELEASE_DEFINITION_ID
          value:
            value: RELEASE_DEFINITION_ID
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ARTIFACTS_PAYLOAD
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_VARIABLES_LIST
          value:
            value: >-
              "{\"feedName\": {\"value\": \"" + PROJECT_NAME + "/" + FEED_NAME +
              "\"},\"packageName\": {\"value\": \"" + (FD_PACKAGE_NAME + "-" +
              PACKAGE_NAME + "-" + FD_ENVIRONMENT_CODE).toLowerCase() +
              "\"},\"packageVersion\": {\"value\": \"" + FINALVERSION + "\"}}"
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROPERTIES
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_MANUAL_ENVIRONMENTS
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_STATUS_CHECK
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PARTIALLY_SUCCEEDED_ENVIRONMENTS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_RESP
        - output: FDAZ_DEVOPS_OUT_RELEASE_WEB_URL
        - output: FDAZ_DEVOPS_OUT_REL_ENV_STAGE_DETAILS
          variable: ENV_DETAILS
        - output: FDAZ_DEVOPS_OUT_RELEASE_ID
          variable: RELEASE_ID
      userInputs: []
      userOutputs: []
  - id: '6'
    name: parseJson
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonXPathPlugin
      pluginOperation: parseJson
      inputs:
        - name: FDP_SOURCE
          value:
            value: ENV_DETAILS
            isExpression: true
          isEncrypted: false
        - name: FDP_XPATH
          value:
            value: '''$..[?(@.name==\''''+FD_ENVIRONMENT_NAME+''\'')].id'''
            isExpression: true
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDP_VALUE
          variable: ENVIRONMENT_ID
      userInputs: []
      userOutputs: []
  - id: '7'
    name: updateReleaseEnvStatus
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: updateReleaseEnvStatus
      inputs:
        - name: FDAZ_DEVOPS_INP_RELEASE_ID
          value:
            value: RELEASE_ID
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_ENVIRONMENT_ID
          value:
            value: ENVIRONMENT_ID.substring(1,ENVIRONMENT_ID.length()-1)
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_ENVIRONMENT_STATUS
          value:
            value: InProgress
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_VARIABLES_LIST
          value:
            value: ENVIRONMENT_VARIABLES_LIST
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_COMMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_SCHEDULED_DEPLOYMENT_TIME
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_REL_ENV_STATUS_CHECK
          value:
            value: 'true'
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PARTIALLY_SUCCEEDED_STATUS_CHECK
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_RESP
        - output: FDAZ_DEVOPS_OUT_REL_ENV_WEB_URL
          variable: RELEASE_URL
      userInputs: []
      userOutputs: []
  - id: '8'
    name: packageExecute
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: packageExecute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: echo "Deployment Completed"
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_PERFORM_PROPERTY_REPLACEMENT
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: true
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []

Project Configuration

Create a Project by choosing the Classification: Package-based and Project Types as Generic

Once it is created the Source Control needs to be configured, so that we can discover the files from the respective repository. This time the source control is the one where we have uploaded the ARM template.

Discover the files using File Catalog tab

Now we can create Package upon selecting specific ARM template which is to be deployed to the Target Synapse Workspace.

Click on the Configuration tab, select General and update the Build and Deploy workflows with respective Target group.

Parameterizing the values

There are many ways to do environment specific parameter value customization. 1) Using Replacement configuration of Project 2) Specifying the parameter as ${{}} in the source json file itself or the generated ARM template 3) Edit the generated ARM template to include the parameter as ARM template parameter. We will show option 2 in the following example.

Replacement configuration on Project

Starting 7.0 we have a new feature that allows us to replace a string in any file with another string based on as it is executed to different environment. Ref: Replacements - FlexDeploy 7.0 - Confluence (atlassian.net)

For our use-case a specific parameter as below is created in the Linked Service:

When this source workspace is integrated with GIT, the linked service json file gets auto-synced there. One can edit the parameter manually to hold the target group property as

Once this is done as shown earlier, the target group property will take care of replacing it with right environment specific value during deployment.

Target Group configuration

Build = for build execution; UAT = for deploy execution.

Build

UAT

The Environment Specific Variables field controls the target workspace name and several other environment specific parameters. To know more follow this link. Follow this link to know about the Azure DevOps Release Pipeline configuration. This must be configured prior to configuring the FlexDeploy workflows and project properties.

Execution

Initiate Build on the selected Package. Once done initiate the Deploy. Build produces the ARM templates as a zip bundle, which Deploy further passes on to the Azure DevOps Release Pipeline to deploy to target workspace.

The Output tab of createRelease step would show the Azue DevOps Release URL to further see the log in more detail:

Post deployment if we verify the Linked Service in UAT workspace, we see the updated values in there.

Flow (Approach B)

Azure Devops pipeline configuration

More information is here

Workflow configuration

Build Workflow:

SynapsePackageBuildJSON

description: SynapsePackageBuildJSON
steps:
  - id: '1'
    name: execute
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: execute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: "# Source directory (parent directory A)\r\nSRC_DIR=$FD_TEMP_DIR/$FD_PROJECT_NAME\r\n\r\n# Target directory (parent directory B)\r\nDEST_DIR=$FD_TEMP_DIR/JSON\r\n\r\nmkdir -p $FD_TEMP_DIR/JSON\r\nchmod -R 777 $FD_TEMP_DIR/JSON\r\n\r\n# Copy .json files while preserving directory structure\r\nfind \"$SRC_DIR\" -type f -name \"*.json\" | while read -r file; do\r\n    # Get the directory path relative to the source\r\n    rel_path=$(dirname \"${file#$SRC_DIR/}\")\r\n    \r\n    # Create the same directory structure in the target\r\n    mkdir -p \"$DEST_DIR/$rel_path\"\r\n    \r\n    # Copy the file to the target directory\r\n    cp \"$file\" \"$DEST_DIR/$rel_path\"\r\ndone\r\n\r\necho \"All JSON files have been copied successfully.\""
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_LOCK
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '2'
    name: publishArtifacts
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: publishArtifacts
      inputs:
        - name: FDAZ_DEVOPS_INP_FEED
          value:
            value: FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: (FD_PACKAGE_NAME + "-" + ARTIFACTS_PACKAGE_NAME).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_VERSION
          value:
            value: PACKAGE_VERSION
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_DESCRIPTION
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_PATH
          value:
            value: FD_TEMP_DIR + "/JSON"
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
      userInputs: []
      userOutputs: []
  - id: '3'
    name: buildPipeline
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: buildPipeline
      inputs:
        - name: FDAZ_DEVOPS_INP_DEFINITION_NAME
          value:
            value: BUILD_DEFINITION_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_BUILD_DEFINITION_ID
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_QUEUE_ID
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_COMMIT_ID
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_BRANCH_NAME
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_VARIABLES_LIST
          value:
            value: >-
              "workspaceName:cicd-demo-test##artifactsPackageName:" +
              (FD_PACKAGE_NAME + "-" + ARTIFACTS_PACKAGE_NAME).toLowerCase() +
              "##feedName:" + PROJECT_NAME + "/" + FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: (FD_PACKAGE_NAME + "-" + PACKAGE_NAME).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
        - output: FDAZ_DEVOPS_OUT_BUILD_PIPELINE_WEB_URL
      userInputs: []
      userOutputs: []
  - id: '4'
    name: downloadArtifacts
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: downloadArtifacts
      inputs:
        - name: FDAZ_DEVOPS_INP_FEED
          value:
            value: FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: (FD_PACKAGE_NAME + "-" + PACKAGE_NAME).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_VERSION
          value:
            value: PACKAGE_VERSION
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_PATH
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
      userInputs: []
      userOutputs: []
  - id: '5'
    name: Bundle ARM Artifacts to Zip
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: execute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: "# Define the name of the zip file to be created\r\nZIP_NAME=$FD_PROJECT_NAME.zip\r\n\r\nmkdir -p $FD_TEMP_DIR/ARMTEMPLATES\r\nfind $FD_ARTIFACTS_DIR -type f -name \"Template*.json\" -exec cp {} $FD_TEMP_DIR/ARMTEMPLATES \\;\r\ncd $FD_TEMP_DIR/ARMTEMPLATES\r\n\r\n# Find and zip all Template*.json files into a single zip file\r\nzip \"$ZIP_NAME\" *\r\nrm -rf $FD_ARTIFACTS_DIR/*\r\ncp $FD_TEMP_DIR/ARMTEMPLATES/$ZIP_NAME $FD_ARTIFACTS_DIR\r\n\r\necho \"Zip created and moved to artifacts.\"\r\n"
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_LOCK
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: true
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []

Deploy Workflow:

Note: Here ${{BLOBWSPACE_TARGET}} is configured in the input json, which is why we are configuring it here as Target properties.

SynapsePackageDeployARM

name: SynapsePackageDeployARM
description: SynapsePackageDeployARM
variables:
  - code: ENV_DETAILS
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: RELEASE_ID
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: ENVIRONMENT_ID
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: RELEASE_URL
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: PKG_NAME
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
  - code: FINALVERSION
    dataType: String
    returnAsOutput: true
    constant: false
    encrypted: false
    scope: LOCAL
steps:
  - id: '1'
    name: execute
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: execute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: "mkdir -p $FD_TEMP_DIR/ARMFiles\r\nunzip $FD_ARTIFACTS_DIR/*.zip -d $FD_TEMP_DIR/ARMFiles"
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_LOCK
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: true
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '2'
    name: propertyReplacement
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonFilePlugin
      pluginOperation: propertyReplacement
      inputs:
        - name: FDFILE_INP_SOURCE_PATH
          value:
            value: FD_TEMP_DIR + "/ARMFiles"
            isExpression: true
          isEncrypted: false
        - name: FDFILE_INP_FILE_FILTER
          value:
            isExpression: false
          isEncrypted: false
        - name: FDFILE_INP_FILE_FILTER_EXCLUDED
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []
  - id: '3'
    name: Assign
    type: ASSIGN
    data:
      from: >-
        PACKAGE_VERSION.substring(0,PACKAGE_VERSION.lastIndexOf(".") +1) +
        FD_WORKFLOW_EXECUTION_ID
      to: FINALVERSION
  - id: '4'
    name: publishArtifacts
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: publishArtifacts
      inputs:
        - name: FDAZ_DEVOPS_INP_FEED
          value:
            value: FEED_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_NAME
          value:
            value: >-
              (FD_PACKAGE_NAME + "-" + PACKAGE_NAME + "-" +
              FD_ENVIRONMENT_CODE).toLowerCase()
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_VERSION
          value:
            value: FINALVERSION
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_DESCRIPTION
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PACKAGE_PATH
          value:
            value: FD_TEMP_DIR + "/ARMFiles"
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_ERR
        - output: FDAZ_DEVOPS_OUT_RESP
      userInputs: []
      userOutputs: []
  - id: '5'
    name: createRelease
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: createRelease
      inputs:
        - name: FDAZ_DEVOPS_INP_RELEASE_DEFINITION_ID
          value:
            value: RELEASE_DEFINITION_ID
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_ARTIFACTS_PAYLOAD
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_VARIABLES_LIST
          value:
            value: >-
              "{\"feedName\": {\"value\": \"" + PROJECT_NAME + "/" + FEED_NAME +
              "\"},\"packageName\": {\"value\": \"" + (FD_PACKAGE_NAME + "-" +
              PACKAGE_NAME + "-" + FD_ENVIRONMENT_CODE).toLowerCase() +
              "\"},\"packageVersion\": {\"value\": \"" + FINALVERSION + "\"}}"
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROPERTIES
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_MANUAL_ENVIRONMENTS
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_STATUS_CHECK
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PARTIALLY_SUCCEEDED_ENVIRONMENTS
          value:
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_RESP
        - output: FDAZ_DEVOPS_OUT_RELEASE_WEB_URL
        - output: FDAZ_DEVOPS_OUT_REL_ENV_STAGE_DETAILS
          variable: ENV_DETAILS
        - output: FDAZ_DEVOPS_OUT_RELEASE_ID
          variable: RELEASE_ID
      userInputs: []
      userOutputs: []
  - id: '6'
    name: parseJson
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonXPathPlugin
      pluginOperation: parseJson
      inputs:
        - name: FDP_SOURCE
          value:
            value: ENV_DETAILS
            isExpression: true
          isEncrypted: false
        - name: FDP_XPATH
          value:
            value: '''$..[?(@.name==\''''+FD_ENVIRONMENT_NAME+''\'')].id'''
            isExpression: true
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDP_VALUE
          variable: ENVIRONMENT_ID
      userInputs: []
      userOutputs: []
  - id: '7'
    name: updateReleaseEnvStatus
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonAzurePlugin
      pluginOperation: updateReleaseEnvStatus
      inputs:
        - name: FDAZ_DEVOPS_INP_RELEASE_ID
          value:
            value: RELEASE_ID
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_ENVIRONMENT_ID
          value:
            value: ENVIRONMENT_ID.substring(1,ENVIRONMENT_ID.length()-1)
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PROJECT_NAME
          value:
            value: PROJECT_NAME
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_ENVIRONMENT_STATUS
          value:
            value: InProgress
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_VARIABLES_LIST
          value:
            value: ENVIRONMENT_VARIABLES_LIST
            isExpression: true
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_RELEASE_COMMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_SCHEDULED_DEPLOYMENT_TIME
          value:
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_REL_ENV_STATUS_CHECK
          value:
            value: 'true'
            isExpression: false
          isEncrypted: false
        - name: FDAZ_DEVOPS_INP_PARTIALLY_SUCCEEDED_STATUS_CHECK
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs:
        - output: FDAZ_DEVOPS_OUT_RESP
        - output: FDAZ_DEVOPS_OUT_REL_ENV_WEB_URL
          variable: RELEASE_URL
      userInputs: []
      userOutputs: []
  - id: '8'
    name: packageExecute
    type: INVOKE_PLUGIN
    data:
      pluginName: FlexagonShellPlugin
      pluginOperation: packageExecute
      inputs:
        - name: FDSHELL_INP_CODE_SNIPPET
          value:
            value: echo "Deployment Completed"
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_STOP_ON_ERROR
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_DISABLE_ECHO
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_RESTRICT_ENVIRONMENT
          value:
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_NO_SECURE_VARIABLES
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
        - name: FDSHELL_INP_PERFORM_PROPERTY_REPLACEMENT
          value:
            value: 'false'
            isExpression: false
          isEncrypted: false
      endpointInstanceOverride:
        isExpression: false
      consumesArtifacts: false
      producesArtifacts: false
      endpointSelection:
        choice: All
      endpointExecution:
        choice: Any
        stopOnError: false
      outputs: []
      userInputs: []
      userOutputs: []

Project Configuration

Create a Project say BuildAndDeploySynapseResources, by choosing the Classification: Package-based and Project Types as Generic

Once it is created the Source Control needs to be configured, so that we can discover the files from the respective repository. Note: These files are from the GIT integrated workspace that we use in Synapse for DEV environment.

For example, this is how the source control in Project looks like:

Project Properties

Note: These are the values from the Azure DevOps Feed, Project Name that you already have configured. We have explained their configuration more here.

Discover the files using File Catalog tab

Now we can create Package upon selecting specific files. For example, here we chose few scripts, pipeline and a linked service to form the package called change-april.

Click on the Configuration tab, select General and map the Build and Deploy workflows along with respective Target group.

Target Group configuration

Build = for build execution; UAT = for deploy execution.

Build

UAT

The Environment Specific Variables field controls the target workspace name and several other environment specific parameters. To know more follow this link.

Execution

Initiate Build on the selected Package change-april.

**Note: The selected JSON files are published to Azure DevOps Feed as Artifacts with the package name as a keyword in it. Example: <packagename>-<fdprojectname>-json-<environmentcode>. Now Microsoft has certain restrictions on the naming of the Azure Artifacts, hence please follow the guide before you name the package that you create in FD or else adjust your workflow to be in compliance. https://learn.microsoft.com/en-us/azure/devops/organizations/settings/naming-restrictions?view=azure-devops

It creates the FD artifacts with the zip bundle of ARM template. Initiate Deploy on the built version.

Clicking on the createRelease step’s Output tab, you would find the release url of Azure DevOps Release pipeline. Navigating to that shows more log:

FlexDeploy 10.0 Documentation

​Objective

​Design

​Flow (Approach A)

​Workflow configuration for generating ARM template and upload to Repository

Build Workflow:

​Project Configuration

​Project properties

​Execution

​Workflow configuration for fetching the ARM template from the repository and initiate the Synapse Release pipeline

Build Workflow:

Deploy Workflow:

​Project Configuration

​Parameterizing the values

Replacement configuration on Project

​Target Group configuration

​Build

​UAT

​Execution

​Flow (Approach B)

​Azure Devops pipeline configuration

​Workflow configuration

Build Workflow:

Deploy Workflow:

​Project Configuration

​Project Properties

​Target Group configuration

​Build

​UAT

​Execution

Objective

Design

Flow (Approach A)

Workflow configuration for generating ARM template and upload to Repository

Project Configuration

Project properties

Execution

Workflow configuration for fetching the ARM template from the repository and initiate the Synapse Release pipeline

Project Configuration

Parameterizing the values

Target Group configuration

Build

UAT

Execution

Flow (Approach B)

Azure Devops pipeline configuration

Workflow configuration

Project Configuration

Project Properties

Target Group configuration

Build

UAT

Execution