Design, build, and maintain Azure DevOps CI/CD pipelines (YAML-first) for Databricks and Azure platform components. · Implement automated deployments for Databricks assets, including notebooks, repos, jobs/workflows, libraries, and configuration across dev/test/prod. · Develop reusable pipeline templates, shared task groups, and standardized “golden path” patterns for engineering teams. · Implement Infrastructure as Code (IaC) for Azure and Databricks resources (e.g., Terraform and/or Bicep/ARM), including environment promotion strategies. · Establish release practices: branching strategy, artifact/versioning, approvals, change control, and rollback/restore patterns. · Integrate secure authentication and secret management with Azure Key Vault, service connections, managed identities/service principals, and least-privilege RBAC. · Automate deployments/integration for related Azure services as needed (commonly including): · ADLS Gen2, Key Vault, Azure Monitor/Log Analytics · Azure Data Factory (if in use), Event Hubs/Service Bus · Azure Functions/Logic Apps (where applicable) · Improve reliability through quality gates: automated tests, linting, policy checks, and deployment validations. · Implement monitoring/alerting for pipeline health and Databricks job outcomes using Azure Monitor, Log Analytics, and pipeline telemetry. · Produce clear documentation: runbooks, pipeline guides, onboarding material, and operational procedures.
Official notificationAny question or remark? just write us a message
If you would like to discuss anything related to payment, account, licensing,
partnerships, or have pre-sales questions, you’re at the right place.