BYOC Data Plane
Bring Your Own Cloud (BYOC) places the PromptQL Enterprise Data Plane in the customer's AWS account, GCP project, or Azure subscription. PromptQL provisions and operates the Data Plane through a customer-approved cloud identity and scoped permissions.
What you need to do
- Prepare the dedicated cloud boundary and confirm the target region.
- Confirm quotas, organization policies, network ranges, availability zones, and the private-connectivity plan.
- Apply the cloud-specific bootstrap in the applicable section below.
- Provide the required cloud identifiers and deployment parameters to PromptQL.
- Keep the cloud account, quotas, networking, and IAM trust available for ongoing management and upgrades.
PromptQL then uses the approved identity to provision and operate the Data Plane in your cloud account. You retain ownership of the account, network controls, audit logs, and the ability to revoke that identity. Availability is a shared responsibility.
An Enterprise agreement is required. Contact sales to begin provisioning.
AWS
AWS prerequisites
- Confirm the Enterprise deployment model and target AWS region with the PromptQL team.
- Confirm the target region is enabled and has sufficient service quotas.
- Record the Data Plane name, domain, availability-zone IDs, network CIDRs, and private-connectivity requirements.
- BYOC requires a dedicated AWS account with administrative access for the initial bootstrap.
AWS network and availability plan
- Select one AWS region and at least two Availability Zones. Use AZ IDs such as
use1-az1, not account-specific names such asus-east-1a. - Reserve a non-overlapping VPC CIDR. A /16 to /20 is recommended. If routable address space is constrained, AWS supports a /21 to /23 VPC CIDR when a separate /16 to /20 Pod IP CIDR is supplied.
- Reserve a non-overlapping Kubernetes Service CIDR. If using a Pod IP CIDR, it must not overlap either the VPC or service CIDR.
- Confirm the network path to each data source: VPC peering, Transit Gateway, VPN, PrivateLink, or another approved private path.
AWS customer tasks
- Prepare a dedicated AWS account and confirm the target region, quotas, availability zones, and network ranges.
- Deploy the CloudFormation bootstrap stack below in your account.
- Wait for the stack to complete and retrieve its Role ARN.
- Provide the Role ARN, region, network plan, and any custom External ID to PromptQL.
- Keep the bootstrap stack and IAM role in place while PromptQL operates the Data Plane.
PromptQL then assumes the scoped role to provision, upgrade, and operate the approved resources in your AWS account.
Detailed AWS setup
The setup involves creating an IAM role in your AWS account that establishes a trust relationship with PromptQL's AWS automation role (PulumiDDNCli). This role will be used to deploy and manage workloads necessary for PromptQL.
AWS setup requirements
- Dedicated AWS Account with administrative access
- AWS CLI installed and configured
AWS_REGIONenvironment variable set to your desired region (e.g.,export AWS_REGION=us-east-1)- Ensure the AWS region where you plan to deploy is enabled in your account
aws account get-regions --region-opt-status-contains ENABLED --query 'Regions[*].RegionName'
Apply the bootstrap
- Copy the following template and save it as
cloudformation.yaml
cloudformation.yaml
Parameters:
ExternalId:
Type: String
Default: hasura-cloud
Description: External ID for the trust relationship with PromptQL
MinLength: 2
MaxLength: 1224
AllowedPattern: "[A-Za-z0-9+=,.@:\\/-]*"
GCPProjectName:
Type: String
Default: hasura-lux
Description: GCP project ID for OIDC provider
MinLength: 1
MaxLength: 255
AllowedPattern: "[a-z0-9-]*"
Resources:
BootstrapRole:
Type: AWS::IAM::Role
Properties:
RoleName: HasuraCloudBYOC
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
AWS: arn:aws:iam::760537944023:role/PulumiDDNCli
Action: sts:AssumeRole
Condition:
StringEquals:
sts:ExternalId: !Ref ExternalId
BootstrapPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: HasuraCloudBYOC
Roles:
- !Ref BootstrapRole
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ec2:DescribeAddresses
- ec2:DescribeAddressesAttribute
- ec2:DescribeAvailabilityZones
- ec2:DescribeInternetGateways
- ec2:DescribeIamInstanceProfileAssociations
- ec2:DescribeImageAttribute
- ec2:DescribeImages
- ec2:DescribeInstanceAttribute
- ec2:DescribeInstances
- ec2:DescribeInstanceTypes
- ec2:DescribeNatGateways
- ec2:DescribeNetworkInterfaces
- ec2:DescribeRegions
- ec2:DescribeRouteTables
- ec2:DescribeSecurityGroupRules
- ec2:DescribeSecurityGroups
- ec2:DescribeSubnets
- ec2:DescribeTags
- ec2:DescribeVpcAttribute
- ec2:DescribeVpcs
- ec2:DescribeVpcEndpointServices
- ec2:DescribeVpcEndpoints
- ec2:DescribeVpcEndpointServiceConfigurations
- ec2:DescribeVolumes
- eks:DeleteAddon
- eks:DescribeAddon
- eks:DescribeCluster
- eks:DescribeNodegroup
- eks:DescribePodIdentityAssociation
- eks:ListClusters
- iam:GetRole
- iam:GetServiceLinkedRoleDeletionStatus
- sqs:GetQueueAttributes
- rds:DescribeDBInstances
- rds:DescribeOrderableDBInstanceOptions
- s3:ListBucket
- s3:ListBucketVersions
- kms:DescribeKey
- kms:GetKeyPolicy
- kms:GetKeyRotationStatus
- kms:ListAliases
- kms:ListResourceTags
- elasticloadbalancing:DescribeCapacityReservation
- elasticloadbalancing:DescribeTargetGroups
- elasticloadbalancing:DescribeLoadBalancers
- elasticloadbalancing:DescribeTargetGroupAttributes
- elasticloadbalancing:DescribeLoadBalancerAttributes
- elasticloadbalancing:DescribeTags
- elasticloadbalancing:DescribeListeners
- elasticloadbalancing:DescribeListenerAttributes
Resource: '*'
- Effect: Allow
Action:
- ec2:AllocateAddress
- ec2:AssociateAddress
- ec2:AssociateRouteTable
- ec2:CreateInternetGateway
- ec2:CreateNatGateway
- ec2:CreateRoute
- ec2:CreateRouteTable
- ec2:CreateSubnet
- ec2:CreateTags
- ec2:CreateVpc
- ec2:CreateNetworkInterface
- ec2:CreateSecurityGroup
- ec2:AuthorizeSecurityGroupEgress
- ec2:AuthorizeSecurityGroupIngress
- ec2:CreateVpcEndpoint
- ec2:CreateVpcEndpointServiceConfiguration
- eks:CreateCluster
- eks:CreateNodegroup
- eks:CreatePodIdentityAssociation
- globalaccelerator:CreateAccelerator
- globalaccelerator:CreateEndpointGroup
- globalaccelerator:CreateListener
- globalaccelerator:TagResource
- sqs:CreateQueue
- sqs:TagQueue
- acm:RequestCertificate
- events:PutRule
- events:TagResource
- iam:CreateOpenIDConnectProvider
- iam:TagOpenIDConnectProvider
- rds:CreateDBSubnetGroup
- rds:CreateDBInstance
- kms:CreateKey
- elasticloadbalancing:CreateTargetGroup
- elasticloadbalancing:CreateLoadBalancer
- elasticloadbalancing:CreateListener
- secretsmanager:CreateSecret
- lambda:CreateFunction
Resource: '*'
Condition:
StringEquals:
aws:RequestTag/Created-By: HasuraCloud
# RunInstances authorizes the source AMI and the existing networking
# resources separately from the instance and volume it creates.
- Effect: Deny
Action:
- ec2:RunInstances
Resource:
- !Sub arn:${AWS::Partition}:ec2:*::image/ami-*
Condition:
StringNotEquals:
ec2:Owner: '832153857469'
- Effect: Allow
Action:
- ec2:RunInstances
Resource:
- !Sub arn:${AWS::Partition}:ec2:*::image/ami-*
Condition:
StringEquals:
ec2:Owner: '832153857469'
- Effect: Allow
Action:
- ec2:RunInstances
Resource:
- !Sub arn:${AWS::Partition}:ec2:*:${AWS::AccountId}:network-interface/*
- !Sub arn:${AWS::Partition}:ec2:*:${AWS::AccountId}:security-group/*
- !Sub arn:${AWS::Partition}:ec2:*:${AWS::AccountId}:subnet/*
Condition:
StringEquals:
aws:ResourceTag/Created-By: HasuraCloud
- Effect: Allow
Action:
- ec2:RunInstances
Resource:
- !Sub arn:${AWS::Partition}:ec2:*:${AWS::AccountId}:instance/*
- !Sub arn:${AWS::Partition}:ec2:*:${AWS::AccountId}:volume/*
Condition:
StringEquals:
aws:RequestTag/Created-By: HasuraCloud
- Effect: Allow
Action:
- ec2:CreateTags
Resource:
- !Sub arn:aws:ec2:*:${AWS::AccountId}:security-group/*
Condition:
StringEquals:
aws:RequestTag/karpenter.sh/discovery: dataplane
- Effect: Allow
Action:
- ec2:DeleteTags
Resource:
- !Sub arn:aws:ec2:*:${AWS::AccountId}:security-group/*
Condition:
StringEquals:
aws:ResourceTag/karpenter.sh/discovery: dataplane
- Effect: Allow
Action:
- ec2:AuthorizeSecurityGroupIngress
- ec2:RevokeSecurityGroupIngress
Resource:
- !Sub arn:aws:ec2:*:${AWS::AccountId}:security-group/*
Condition:
StringEquals:
aws:ResourceTag/aws:eks:cluster-name: dataplane
- Effect: Allow
Action:
- eks:AssociateAccessPolicy
- eks:DisassociateAccessPolicy
Resource:
- !Sub arn:aws:eks:*:${AWS::AccountId}:access-entry/dataplane/*
- Effect: Allow
Action:
- iam:AttachRolePolicy
- iam:AddRoleToInstanceProfile
- iam:CreateInstanceProfile
- iam:CreatePolicy
- iam:CreateRole
- iam:CreatePolicyVersion
- iam:DeleteInstanceProfile
- iam:DeleteOpenIDConnectProvider
- iam:DeletePolicy
- iam:DeleteRolePolicy
- iam:DeleteRole
- iam:DeleteServiceLinkedRole
- iam:DetachRolePolicy
- iam:UpdateAssumeRolePolicy
- iam:UpdateOpenIDConnectProviderThumbprint
- iam:GetInstanceProfile
- iam:GetOpenIDConnectProvider
- iam:GetPolicy
- iam:GetPolicyVersion
- iam:GetRolePolicy
- iam:ListAttachedRolePolicies
- iam:ListInstanceProfileTags
- iam:ListInstanceProfilesForRole
- iam:ListOpenIDConnectProviderTags
- iam:ListPolicyVersions
- iam:ListRoleTags
- iam:ListRolePolicies
- iam:PassRole
- iam:PutRolePolicy
- iam:RemoveRoleFromInstanceProfile
- iam:TagInstanceProfile
- iam:TagOpenIDConnectProvider
- iam:TagPolicy
- iam:TagRole
- iam:UntagInstanceProfile
- iam:UntagRole
- iam:UpdateRole
- iam:UpdateRoleDescription
Resource:
# Roles
- !Sub arn:aws:iam::${AWS::AccountId}:role/KarpenterNodeRole
- !Sub arn:aws:iam::${AWS::AccountId}:role/KarpenterSandboxNodeRole
- !Sub arn:aws:iam::${AWS::AccountId}:role/eksClusterRole-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/lb-controller-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/autoscaler-controller-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/global-accelerator-operator-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/karpenter-controller-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/HasuraWorkloadAutomationRole-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/vpc-cni-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/ebsCsiDriverRole-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/kms-control-plane-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/bee-control-plane-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/bee-data-plane-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/promptql-artifact-server-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/dataplane-eks-api-lambda-role-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/promptql-warehouse-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/promptql-sandbox-broker-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/promptql-sandbox-base-publisher-*
- !Sub arn:aws:iam::${AWS::AccountId}:role/pql-fc-*
# Instance Profiles
- !Sub arn:aws:iam::${AWS::AccountId}:instance-profile/dataplane_*
- !Sub arn:aws:iam::${AWS::AccountId}:instance-profile/pql-fc-*
# Policies
- !Sub arn:aws:iam::${AWS::AccountId}:policy/lb-controller-*
- !Sub arn:aws:iam::${AWS::AccountId}:policy/autoscaler-controller-*
- !Sub arn:aws:iam::${AWS::AccountId}:policy/global-accelerator-operator-*
- !Sub arn:aws:iam::${AWS::AccountId}:policy/karpenter-controller-*
- !Sub arn:aws:iam::${AWS::AccountId}:policy/dataplane-eks-api-register-targets-*
- !Sub arn:aws:iam::${AWS::AccountId}:policy/dataplane-eks-api-deregister-targets-*
# OIDC Providers
- !Sub arn:aws:iam::${AWS::AccountId}:oidc-provider/oidc.eks.*
- !Sub arn:aws:iam::${AWS::AccountId}:oidc-provider/container.googleapis.com/v1/projects/${GCPProjectName}/*
# Service Roles
- !Sub arn:aws:iam::${AWS::AccountId}:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot
- Effect: Allow
Action:
- iam:CreateServiceLinkedRole
Resource: '*'
Condition:
StringEquals:
iam:AWSServiceName:
- spot.amazonaws.com
- globalaccelerator.amazonaws.com
- eks.amazonaws.com
- eks-nodegroup.amazonaws.com
- rds.amazonaws.com
- Effect: Allow
Action:
- ec2:*
- eks:*
- globalaccelerator:*
- sqs:*
- acm:*
- events:*
- rds:*
- s3:*
- kms:*
- elasticloadbalancing:*
- lambda:*
Resource: '*'
Condition:
StringEquals:
aws:ResourceTag/Created-By: HasuraCloud
- Effect: Allow
Action:
- secretsmanager:DeleteSecret
- secretsmanager:DescribeSecret
- secretsmanager:GetResourcePolicy
- secretsmanager:GetSecretValue
- secretsmanager:ListSecretVersionIds
- secretsmanager:PutSecretValue
- secretsmanager:RestoreSecret
- secretsmanager:TagResource
- secretsmanager:UntagResource
- secretsmanager:UpdateSecret
- secretsmanager:UpdateSecretVersionStage
Resource:
- !Sub arn:${AWS::Partition}:secretsmanager:*:${AWS::AccountId}:secret:pql-fc-*
- Effect: Allow
Action:
- s3:Get*
- s3:CreateBucket
- s3:PutBucketPublicAccessBlock
- s3:PutBucketVersioning
- s3:PutBucketPolicy
- s3:PutBucketTagging
- s3:DeleteBucketPolicy
- s3:DeleteObjectVersion
- s3:DeleteBucket
Resource:
- arn:aws:s3:::metadata-store-*
- arn:aws:s3:::promptql-store-*
- arn:aws:s3:::promptql-warehouse-*
- arn:aws:s3:::promptql-sandbox-*
- Effect: Allow
Action:
- kms:CreateAlias
- kms:DeleteAlias
- kms:UpdateAlias
Resource:
- !Sub arn:aws:kms:*:${AWS::AccountId}:alias/bee
- !Sub arn:aws:kms:*:${AWS::AccountId}:alias/pql-fc-*
# The publisher account grants this role access to the encrypted AMI
# key. Cross-account use also needs an identity policy in this account.
- Effect: Allow
Action:
- kms:Decrypt
- kms:DescribeKey
- kms:GenerateDataKey
- kms:GenerateDataKeyWithoutPlaintext
- kms:ReEncryptFrom
- kms:ReEncryptTo
Resource:
- !Sub arn:${AWS::Partition}:kms:*:832153857469:key/*
- Effect: Allow
Action:
- kms:CreateGrant
Resource:
- !Sub arn:${AWS::Partition}:kms:*:832153857469:key/*
Condition:
Bool:
kms:GrantIsForAWSResource: 'true'
- Effect: Allow
Action:
- vpce:AllowMultiRegion
Resource: '*'
Outputs:
RoleArn:
Description: "ARN of the HasuraCloudBYOC IAM Role"
Value: !GetAtt BootstrapRole.Arn
What the AWS role permits
The HasuraCloudBYOC role grants PromptQL the permissions required to deploy and manage the AWS resources for your PromptQL Data Plane, including the Firecracker Host fleet. Permissions are limited to the actions and resource patterns defined in the policy above.
-
Deploy the CloudFormation stack:
First, check if the stack already exists:
aws cloudformation describe-stacks --stack-name hasura-cloud-byocThen, based on the result:
-
If you see an error "Stack with id hasura-cloud-byoc does not exist":
# Create new stack aws cloudformation create-stack \ --stack-name hasura-cloud-byoc \ --template-body file://cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --parameters ParameterKey=ExternalId,ParameterValue=hasura-cloud # Wait for creation to complete aws cloudformation wait stack-create-complete \ --stack-name hasura-cloud-byoc -
If the stack exists:
# Update existing stack aws cloudformation update-stack \ --stack-name hasura-cloud-byoc \ --template-body file://cloudformation.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --parameters ParameterKey=ExternalId,ParameterValue=hasura-cloud # Wait for update to complete aws cloudformation wait stack-update-complete \ --stack-name hasura-cloud-byoc
-
-
Monitor stack status:
aws cloudformation describe-stacks \ --stack-name hasura-cloud-byoc \ --query 'Stacks[0].StackStatus' -
Once complete, retrieve the Role ARN:
aws cloudformation describe-stacks \ --stack-name hasura-cloud-byoc \ --query 'Stacks[0].Outputs[?OutputKey==`RoleArn`].OutputValue' \ --output text
AWS deployment details to provide
Share the following with the PromptQL team:
- (Required) Role ARN (From output above)
- (Required) AWS Region
- (Optional) External ID
- The external ID used in the trust relationship between your AWS account and PromptQL's AWS account
- This is the value you specified for the
ExternalIdparameter in the CloudFormation template - If not specified, the default value "hasura-cloud" will be used
- Must have a minimum of 2 characters and a maximum of 1,224 characters
- Must be alphanumeric without white space, but can include the following symbols: plus (+), equal (=), comma (,), period (.), at (@), colon (:), forward slash (/), and hyphen (-)
- Important: Make sure to provide this value to the PromptQL team if you've customized it
- (Optional) Preferred Availability Zones
- Use AZ IDs (e.g., use1-az1, use1-az2) instead of AZ names (us-east-1a, us-east-1b)
- You can get the AZ IDs by running:
aws ec2 describe-availability-zones \ --region <region> \ --output table \ --query "AvailabilityZones[?State=='available'] | [].{ZoneName: ZoneName, ZoneId: ZoneId}" - If you have specific zones which you'd like to use, please pass it along. Otherwise, PromptQL will assign accordingly.
- (Optional) VPC CIDR (/16-/19 CIDR)
- If you have a specific CIDR in mind for the VPC setup, please pass it along. If not specified, PromptQL will assign 10.0.0.0/16.
- A /21-/23 VPC CIDR is also supported if you provide a Pod IP CIDR (see below), which enables VPC CNI custom networking.
- Note: If you are planning to use VPC Peering, this CIDR should not conflict with any networks on your side.
- (Optional) Kubernetes Service CIDR (/16-20 CIDR)
- A /16-/20 CIDR block used for Kubernetes service cluster IP addresses in your Data Plane. If not specified, PromptQL will assign 172.20.0.0/16.
- Note: If you are planning to use VPC Peering, this CIDR should not conflict with any networks on your side.
- (Optional) Pod IP CIDR (/16-/20 CIDR)
- A /16-/20 CIDR block that Kubernetes pods draw IP addresses from, via AWS VPC CNI custom networking. When set, the VPC CIDR can be as small as /23. If not specified, PromptQL will assign 100.64.0.0/16.
- Optional for a /16-/20 VPC CIDR; required for a /21-/23 VPC CIDR. Must not overlap the VPC CIDR or the Kubernetes Service CIDR.
- Note: If you are planning to use VPC Peering, this CIDR should not conflict with any networks on your side.
GCP
GCP prerequisites
- Confirm the Enterprise deployment model and target GCP region with the PromptQL team.
- Record the Data Plane name, domain, zones, network CIDR, and private-connectivity requirements.
- BYOC requires a dedicated GCP project with billing enabled and permission to create project-level custom roles and IAM bindings.
GCP network and availability plan
- Select one GCP region and at least two zones.
- Reserve a non-overlapping VPC CIDR, normally /16 to /20.
- Confirm the network path to each data source: VPC Network Peering, VPN, Private Service Connect, or another approved private path.
- For BYOC, use a dedicated GCP project and confirm organization policies and quotas permit the required GKE, Cloud SQL, storage, load-balancing, KMS, and networking resources.
GCP customer tasks
- Prepare a dedicated GCP project with billing enabled and confirm the target region, quotas, organization policies, and network range.
- Apply the Terraform configuration below as a project administrator.
- Verify that all 15 custom roles and their IAM bindings were created, or 16 with
create_argocd_custom_role = true. - Provide the project ID, region, and network plan to PromptQL.
- Keep the custom roles, IAM bindings, required APIs, and project quotas available while PromptQL operates the Data Plane.
PromptQL then uses its scoped provisioning service account to provision, upgrade, and operate the approved resources in your GCP project.
Detailed GCP setup
The setup involves enabling APIs and creating IAM policy bindings. The IAM policy bindings establish a trust relationship with PromptQL's GCP service account, which is used to deploy and manage the infrastructure required for PromptQL.
GCP setup requirements
- Dedicated GCP Project
Apply the setup
- Create the project-level custom IAM roles for PromptQL's
ddn-automationservice account and bind them (see below). These grant only the granular permissions PromptQL's provisioning automation uses, avoiding the broader capabilities bundled into predefined roles such ascloudsql.adminorstorage.admin.
Keep this configuration up to date
New PromptQL features may require additional GCP permissions. When PromptQL provides an updated version of this Terraform configuration for a new feature, review its
terraform planand apply it to your GCP project before enabling that feature. This updates the custom roles and IAM bindings with the permissions the feature requires.If you see
PERMISSION_DENIEDafter applying the latest configuration, contact PromptQL support rather than changing the custom roles manually.
Copy the two files below into an empty directory as main.tf and argocd_role.tf, then apply
them together. They are the GCP counterpart to the AWS CloudFormation template above. A single
terraform apply enables the required APIs, creates 15 custom roles (16 with
create_argocd_custom_role = true), binds them to ddn-automation, and applies the IAM
grant-restriction condition. argocd_role.tf additionally defines the optional least-privilege
ArgoCD reconciler role, which is off by default; main.tf references values from it, so keep both
files even if you do not enable that role. When PromptQL provides a newer version, replace both
local files, review the resulting terraform plan, and apply it.
main.tf
# GCP BYOC: least-privilege IAM for PromptQL.
#
# Applied by a project admin in their own project, together with argocd_role.tf. It:
# 1. enables the APIs PromptQL's provisioning automation needs,
# 2. creates 15 custom least-privilege project roles (replacing the broad predefined
# roles), or 16 with create_argocd_custom_role = true,
# 3. binds them to PromptQL's `ddn-automation` service account,
# 4. applies an IAM grant-restriction condition on the five roles that carry a *.setIamPolicy
# permission, so `ddn-automation` may only grant the specific roles the provision hands out.
#
# Adding a permission later = edit the relevant role's list below and `terraform apply`. Widening
# what the automation may grant = add the role to the relevant role's `grantable_roles`.
terraform {
required_version = ">= 1.3"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.0, < 7.0"
}
}
}
provider "google" {
project = var.project_id
}
locals {
member = "serviceAccount:${var.ddn_automation_service_account}"
# APIs the data-plane project must have enabled for provisioning to run.
required_apis = [
"compute.googleapis.com",
"container.googleapis.com",
"dns.googleapis.com",
"servicenetworking.googleapis.com",
"sqladmin.googleapis.com",
"cloudkms.googleapis.com",
"certificatemanager.googleapis.com",
"gkehub.googleapis.com",
"multiclusterservicediscovery.googleapis.com",
"multiclusteringress.googleapis.com",
"trafficdirector.googleapis.com",
"monitoring.googleapis.com",
"storage.googleapis.com",
"iam.googleapis.com",
"iamcredentials.googleapis.com",
"cloudresourcemanager.googleapis.com", # google_project_iam_member goes through Resource Manager
"secretmanager.googleapis.com", # Firecracker Host bootstrap-trust secret
]
# Each setIamPolicy-bearing role below carries its OWN `grantable_roles` allowlist, scoped to
# exactly the roles ddn-automation grants at that resource level (project / SA / bucket / key).
# The grant-restriction condition on each binding is built from that per-role list: so e.g. the
# project-IAM role can grant only its project-level roles, never storage/KMS roles.
# 15 custom roles. A role with `grants_iam = true` also defines `grantable_roles`: its binding
# gets a grant-restriction condition permitting ONLY those roles at that scope (see below).
custom_roles = {
compute_network_admin = {
role_id = "promptqlDdnComputeNetworkAdmin"
title = "PromptQL DDN - Compute Network Admin"
description = "Least-privilege replacement for roles/compute.networkAdmin, scoped to PromptQL provisioning"
permissions = [
"compute.networks.create",
"compute.networks.get",
"compute.networks.list",
"compute.networks.update",
"compute.networks.delete",
"compute.networks.use",
"compute.networks.updatePolicy", # add subnets/routers/PSA peering to the VPC
"compute.subnetworks.create",
"compute.subnetworks.get",
"compute.subnetworks.list",
"compute.subnetworks.update",
"compute.subnetworks.delete",
"compute.subnetworks.use",
"compute.subnetworks.setPrivateIpGoogleAccess",
"compute.routers.create",
"compute.routers.get",
"compute.routers.list",
"compute.routers.update", # also covers RouterNat create/delete (a router PATCH)
"compute.routers.delete",
"compute.routers.use",
# Firewall rules for the Firecracker Host relay port, scoped to the Host
# service account.
"compute.firewalls.create",
"compute.firewalls.get",
"compute.firewalls.list",
"compute.firewalls.update",
"compute.firewalls.delete",
"compute.addresses.create",
"compute.addresses.get",
"compute.addresses.list",
"compute.addresses.delete",
"compute.addresses.use",
# Regional INTERNAL addresses: the Firecracker shared LB VIP and each
# Host's reserved relay IP.
"compute.addresses.createInternal",
"compute.addresses.useInternal",
"compute.addresses.deleteInternal",
"compute.addresses.setLabels",
"compute.globalAddresses.create", # external gateway IP
"compute.globalAddresses.get",
"compute.globalAddresses.list",
"compute.globalAddresses.delete",
"compute.globalAddresses.use",
"compute.globalAddresses.createInternal", # internal PSA range (AddressType=INTERNAL)
"compute.globalAddresses.deleteInternal", # tear down the PSA range on destroy
"compute.globalOperations.get",
"compute.globalOperations.list",
"compute.regionOperations.get",
"compute.regionOperations.list",
"compute.projects.get",
"servicenetworking.services.addPeering",
"servicenetworking.services.get",
"servicenetworking.services.deleteConnection", # tear down the PSA peering on destroy
"servicenetworking.operations.get",
]
}
dns_admin = {
role_id = "promptqlDdnDnsAdmin"
title = "PromptQL DDN - DNS Admin"
description = "Least-privilege replacement for roles/dns.admin, scoped to PromptQL provisioning"
permissions = [
"dns.managedZones.create",
"dns.managedZones.get",
"dns.managedZones.list",
"dns.managedZones.update",
"dns.managedZones.delete",
"dns.resourceRecordSets.create",
"dns.resourceRecordSets.get",
"dns.resourceRecordSets.list",
"dns.resourceRecordSets.update",
"dns.resourceRecordSets.delete",
"dns.changes.create",
"dns.changes.get",
"dns.changes.list",
"dns.managedZoneOperations.get",
"dns.managedZoneOperations.list",
"dns.projects.get",
"dns.networks.bindPrivateDNSZone", # bind the private managed zone to the VPC
]
}
gkehub_editor = {
role_id = "promptqlDdnGkeHubEditor"
title = "PromptQL DDN - GKE Hub Editor"
description = "Least-privilege replacement for roles/gkehub.editor, scoped to PromptQL provisioning"
permissions = [
"gkehub.features.create",
"gkehub.features.get",
"gkehub.features.list",
"gkehub.features.update",
"gkehub.features.delete",
"gkehub.memberships.create",
"gkehub.memberships.get",
"gkehub.memberships.list",
"gkehub.memberships.update",
"gkehub.memberships.delete",
"gkehub.operations.get",
"gkehub.operations.list",
"gkehub.locations.get",
"gkehub.locations.list",
]
}
container_cluster_admin = {
role_id = "promptqlDdnContainerClusterAdmin"
title = "PromptQL DDN - Container Cluster Admin"
description = "Least-privilege replacement for roles/container.clusterAdmin, scoped to PromptQL provisioning"
permissions = [
"container.clusters.create",
"container.clusters.get",
"container.clusters.list",
"container.clusters.update",
"container.clusters.delete",
"container.operations.get",
"container.operations.list",
"resourcemanager.projects.get",
"compute.instanceGroupManagers.get",
"compute.instanceGroupManagers.list",
"compute.instanceGroups.get",
"compute.instanceGroups.list",
"compute.instances.get",
"compute.instances.list",
]
}
monitoring_metrics_scopes_admin = {
role_id = "promptqlDdnMonitoringMetricsScopesAdmin"
title = "PromptQL DDN - Monitoring Metrics Scopes Admin"
description = "Least-privilege replacement for roles/monitoring.metricsScopesAdmin, scoped to PromptQL provisioning"
permissions = [
"monitoring.metricsScopes.link",
"resourcemanager.projects.get",
]
}
certificate_manager_editor = {
role_id = "promptqlDdnCertificateManagerEditor"
title = "PromptQL DDN - Certificate Manager Editor"
description = "Least-privilege replacement for roles/certificatemanager.editor, scoped to PromptQL provisioning"
permissions = [
"certificatemanager.certmaps.create",
"certificatemanager.certmaps.get",
"certificatemanager.certmaps.list",
"certificatemanager.certmaps.update",
"certificatemanager.certmaps.delete",
"certificatemanager.certmapentries.create",
"certificatemanager.certmapentries.get",
"certificatemanager.certmapentries.list",
"certificatemanager.certmapentries.update",
"certificatemanager.certmapentries.delete",
"certificatemanager.certs.create",
"certificatemanager.certs.get",
"certificatemanager.certs.list",
"certificatemanager.certs.update",
"certificatemanager.certs.delete",
"certificatemanager.dnsauthorizations.create",
"certificatemanager.dnsauthorizations.get",
"certificatemanager.dnsauthorizations.list",
"certificatemanager.dnsauthorizations.update",
"certificatemanager.dnsauthorizations.delete",
"certificatemanager.locations.get",
"certificatemanager.locations.list",
"certificatemanager.operations.get",
"certificatemanager.operations.list",
# .use perms: the SA itself creates the Certificate referencing the DnsAuthorization and
# the CertificateMapEntry referencing the cert during provision (not at GKE runtime).
"certificatemanager.dnsauthorizations.use",
"certificatemanager.certs.use",
"certificatemanager.certmaps.use",
]
}
project_iam_admin = {
role_id = "promptqlDdnProjectIamAdmin"
title = "PromptQL DDN - Project IAM Admin"
description = "Least-privilege replacement for roles/resourcemanager.projectIamAdmin, scoped to PromptQL provisioning"
grants_iam = true # resourcemanager.projects.setIamPolicy
# Only the project-level roles the provision grants; never storage/KMS at project scope.
# The argocd entry is variable: `roles/container.admin` and/or the custom reconciler
# role, per the two flags in argocd_role.tf.
grantable_roles = concat(
[
"roles/container.defaultNodeServiceAccount", # -> GKE node SA
"roles/compute.networkViewer", # -> gke-mcs importer
# -> FC sandbox Broker (host fence). A project custom role the fleet
# provisioning creates at runtime; bound at project scope to the
# Broker's workload-identity principal.
"projects/${var.project_id}/roles/sandboxFirecrackerBrokerFence",
],
local.argocd_grantable_roles, # -> argocd
)
permissions = [
"resourcemanager.projects.getIamPolicy",
"resourcemanager.projects.setIamPolicy",
]
}
service_account_admin = {
role_id = "promptqlDdnServiceAccountAdmin"
title = "PromptQL DDN - Service Account Admin"
description = "Least-privilege replacement for roles/iam.serviceAccountAdmin, scoped to PromptQL provisioning"
grants_iam = true # iam.serviceAccounts.setIamPolicy
# Only the actAs grant on the freshly-created GKE node SA.
grantable_roles = ["roles/iam.serviceAccountUser"]
permissions = [
"iam.serviceAccounts.create",
"iam.serviceAccounts.get",
"iam.serviceAccounts.list",
"iam.serviceAccounts.update",
"iam.serviceAccounts.delete",
"iam.serviceAccounts.getIamPolicy",
"iam.serviceAccounts.setIamPolicy",
]
}
cloudsql_admin = {
role_id = "promptqlDdnCloudsqlAdmin"
title = "PromptQL DDN - Cloud SQL Admin"
description = "Least-privilege replacement for roles/cloudsql.admin, scoped to PromptQL provisioning"
# Excludes cloudsql.instances.connect/export/import: the automation provisions instances,
# databases and users but never connects to or reads a customer database.
permissions = [
"cloudsql.instances.create",
"cloudsql.instances.get",
"cloudsql.instances.list",
"cloudsql.instances.update",
"cloudsql.instances.delete",
"cloudsql.databases.create",
"cloudsql.databases.get",
"cloudsql.databases.list",
"cloudsql.databases.update",
"cloudsql.databases.delete",
"cloudsql.users.create",
"cloudsql.users.get",
"cloudsql.users.list",
"cloudsql.users.update",
"cloudsql.users.delete",
]
}
workload_identity_pool_admin = {
role_id = "promptqlDdnWorkloadIdentityPoolAdmin"
title = "PromptQL DDN - Workload Identity Pool Admin"
description = "Least-privilege replacement for roles/iam.workloadIdentityPoolAdmin, scoped to PromptQL provisioning"
# Workload Identity Federation permissions use the service-qualified form in custom roles.
permissions = [
"iam.googleapis.com/workloadIdentityPools.create",
"iam.googleapis.com/workloadIdentityPools.get",
"iam.googleapis.com/workloadIdentityPools.list",
"iam.googleapis.com/workloadIdentityPools.update",
"iam.googleapis.com/workloadIdentityPools.delete",
"iam.googleapis.com/workloadIdentityPoolProviders.create",
"iam.googleapis.com/workloadIdentityPoolProviders.get",
"iam.googleapis.com/workloadIdentityPoolProviders.list",
"iam.googleapis.com/workloadIdentityPoolProviders.update",
"iam.googleapis.com/workloadIdentityPoolProviders.delete",
]
}
storage_admin = {
role_id = "promptqlDdnStorageAdmin"
title = "PromptQL DDN - Storage Admin"
description = "Least-privilege replacement for roles/storage.admin, scoped to PromptQL provisioning"
grants_iam = true # storage.buckets.setIamPolicy
# Only the bucket-level roles the provision grants to workloads (on buckets, never project-wide).
grantable_roles = [
"roles/storage.objectAdmin",
"roles/storage.objectViewer",
"roles/storage.legacyBucketReader",
"projects/${var.project_id}/roles/sandboxFirecrackerHostObjects",
"projects/${var.project_id}/roles/sandboxFirecrackerHostObjectsRead",
"projects/${var.project_id}/roles/sandboxFirecrackerArtifactPublisher",
]
# storage.objects.list/delete are needed because buckets are created with force_destroy,
# which empties objects before deleting the bucket; the automation never reads object content.
permissions = [
"storage.buckets.create",
"storage.buckets.get",
"storage.buckets.list",
"storage.buckets.update",
"storage.buckets.delete",
"storage.buckets.getIamPolicy",
"storage.buckets.setIamPolicy",
"storage.objects.list",
"storage.objects.delete",
]
}
cloudkms_admin = {
role_id = "promptqlDdnCloudkmsAdmin"
title = "PromptQL DDN - Cloud KMS Admin"
description = "Least-privilege replacement for roles/cloudkms.admin, scoped to PromptQL provisioning"
grants_iam = true # cloudkms.cryptoKeys.setIamPolicy
# Only the key-level roles the provision grants on the bee key (on the key, never project-wide).
grantable_roles = [
"roles/cloudkms.cryptoKeyDecrypter",
"roles/cloudkms.cryptoKeyEncrypterDecrypter",
]
# Key rings and keys cannot be deleted in GCP, so on `pulumi destroy` the provider lists and
# destroys the key's versions instead (cryptoKeyVersions.list/destroy). Still excludes
# encrypt/decrypt: the automation never performs cryptographic operations itself.
permissions = [
"cloudkms.keyRings.create",
"cloudkms.keyRings.get",
"cloudkms.keyRings.list",
"cloudkms.cryptoKeys.create",
"cloudkms.cryptoKeys.get",
"cloudkms.cryptoKeys.list",
"cloudkms.cryptoKeys.update",
"cloudkms.cryptoKeys.getIamPolicy",
"cloudkms.cryptoKeys.setIamPolicy",
"cloudkms.cryptoKeyVersions.list", # teardown: enumerate versions to destroy
"cloudkms.cryptoKeyVersions.destroy", # teardown: destroy versions (key itself can't be deleted)
]
}
# --- Firecracker/v2 sandbox Host fleet -----------------------------------
# See the Firecracker Host fleet note below. The fleet's networking
# permissions live in compute_network_admin above.
compute_instance_admin = {
role_id = "promptqlDdnComputeInstanceAdmin"
title = "PromptQL DDN - Compute Instance Admin"
description = "Least-privilege replacement for roles/compute.instanceAdmin.v1, scoped to the Firecracker sandbox Host VMs"
# No stop/start: the automation never updates a running Host in place.
permissions = [
"compute.instances.create",
"compute.instances.get",
"compute.instances.list",
"compute.instances.delete",
"compute.instances.setMetadata",
"compute.instances.setTags",
"compute.instances.setLabels",
"compute.instances.setServiceAccount",
"compute.disks.create", # boot disk from the shared Host image
"compute.zoneOperations.get", # poll the zonal insert/delete operation
"compute.zoneOperations.list",
]
}
iam_role_admin = {
role_id = "promptqlDdnIamRoleAdmin"
title = "PromptQL DDN - IAM Role Admin"
description = "Least-privilege replacement for roles/iam.roleAdmin, scoped to the custom object-store/fence roles the Firecracker fleet provisioning creates"
# Creates the fleet's custom roles but cannot grant them: binding stays
# with the grant-restricted project_iam_admin / storage_admin.
permissions = [
"iam.roles.create",
"iam.roles.get",
"iam.roles.update",
"iam.roles.delete",
"iam.roles.undelete",
]
}
secret_manager_admin = {
role_id = "promptqlDdnSecretManagerAdmin"
title = "PromptQL DDN - Secret Manager Admin"
description = "Least-privilege replacement for roles/secretmanager.admin, scoped to the Firecracker Host bootstrap-trust secret"
grants_iam = true # secretmanager.secrets.setIamPolicy
# Granted to the Host service account on the bootstrap-trust secret only.
grantable_roles = ["roles/secretmanager.secretAccessor"]
# Excludes secretmanager.versions.access: the automation writes the secret
# but never reads it back.
permissions = [
"secretmanager.secrets.create",
"secretmanager.secrets.get",
"secretmanager.secrets.update",
"secretmanager.secrets.delete",
"secretmanager.secrets.getIamPolicy",
"secretmanager.secrets.setIamPolicy",
"secretmanager.versions.add",
"secretmanager.versions.get",
"secretmanager.versions.enable",
"secretmanager.versions.destroy", # teardown / rotation: destroy old versions
]
}
}
}
# 1. Enable required APIs (disable_on_destroy=false so teardown never disables a shared API).
resource "google_project_service" "apis" {
for_each = toset(local.required_apis)
project = var.project_id
service = each.value
disable_on_destroy = false
disable_dependent_services = false
}
# 2. Create the 15 custom roles.
resource "google_project_iam_custom_role" "roles" {
for_each = local.custom_roles
project = var.project_id
role_id = each.value.role_id
title = each.value.title
description = each.value.description
permissions = each.value.permissions
stage = "GA"
# iam.googleapis.com (which backs custom roles) is enabled by this same module, so the roles
# must wait for API enablement: otherwise the first apply on a fresh project can race it.
depends_on = [google_project_service.apis]
}
# 3a. Bind roles that do NOT grant IAM to others (no grant-restriction condition needed).
resource "google_project_iam_member" "bindings" {
for_each = { for k, v in local.custom_roles : k => v if !try(v.grants_iam, false) }
project = var.project_id
role = google_project_iam_custom_role.roles[each.key].id
member = local.member
}
# 3b. Bind the five roles that carry a setIamPolicy permission, WITH the grant-restriction
# condition. modifiedGrantsByRole is only populated on setIamPolicy calls, so non-grant
# operations (create/get/delete/etc.) see an empty set and pass the condition unconditionally.
resource "google_project_iam_member" "iam_granting_bindings" {
for_each = { for k, v in local.custom_roles : k => v if try(v.grants_iam, false) }
project = var.project_id
role = google_project_iam_custom_role.roles[each.key].id
member = local.member
condition {
title = "Restrict IAM Granting for ddn-automation"
description = "Roles ddn-automation may grant via ${each.value.role_id}"
expression = "api.getAttribute(\"iam.googleapis.com/modifiedGrantsByRole\", []).hasOnly(${jsonencode(each.value.grantable_roles)})"
}
lifecycle {
precondition {
condition = var.allow_container_admin_grant || var.create_argocd_custom_role
error_message = "ArgoCD needs one of the two roles: set create_argocd_custom_role = true before setting allow_container_admin_grant = false, otherwise ddn-automation cannot grant the reconciler any role and provisioning fails."
}
}
}
variable "project_id" {
type = string
description = "The GCP project ID where the PromptQL data plane will be provisioned."
}
variable "ddn_automation_service_account" {
type = string
description = "PromptQL's provisioning service account the custom roles are bound to."
default = "ddn-automation@hasura-ddn.iam.gserviceaccount.com"
validation {
condition = can(regex("^[^@]+@[^.]+\\.iam\\.gserviceaccount\\.com$", var.ddn_automation_service_account))
error_message = "ddn_automation_service_account must be a GCP service account email (name@project.iam.gserviceaccount.com)."
}
}
variable "create_argocd_custom_role" {
type = bool
description = <<-EOT
Create the least-privilege `promptqlDdnArgocdReconciler` role for PromptQL's ArgoCD
reconciler and allow `ddn-automation` to grant it, instead of `roles/container.admin`.
Pass the resulting `argocd_custom_role_name` output to PromptQL so the data plane is
provisioned against it.
EOT
default = false
}
variable "argocd_private_endpoint_access" {
type = bool
description = <<-EOT
Include `container.clusters.connect` in the ArgoCD reconciler role. Default false: with a
public GKE control-plane endpoint (every PromptQL data plane today) the reconciler reaches
the API server on its IP endpoint with the cluster CA and does not need this permission.
Set to true only if PromptQL tells you your data plane will run a **private** control-plane
endpoint, which is reached over GKE's DNS endpoint (`*.gke.goog`) and authorizes on exactly
this permission. Only has an effect when create_argocd_custom_role is true.
EOT
default = false
}
variable "allow_container_admin_grant" {
type = bool
description = <<-EOT
Keep `roles/container.admin` in `ddn-automation`'s grant-restriction allowlist. Default
true (the historical behavior). Set to false only once the ArgoCD service account has
been moved onto the custom role: the automation then cannot grant `container.admin` at
project level at all. During the cutover itself, leave this true alongside
`create_argocd_custom_role`, because the condition also gates the *removal* of the old
binding.
EOT
default = true
}
output "argocd_custom_role_name" {
description = "Full resource name of the ArgoCD reconciler role to hand to PromptQL, or null when create_argocd_custom_role is false."
value = var.create_argocd_custom_role ? local.argocd_custom_role_name : null
}
argocd_role.tf
# Optional: a least-privilege custom role for PromptQL's ArgoCD reconciler.
#
# PromptQL's GitOps controller (`argocd@...`) needs Kubernetes RBAC write on the data-plane
# cluster, which among the predefined GKE roles only `roles/container.admin` carries.
# `container.admin` additionally carries GKE cluster lifecycle (create/update/delete,
# getCredentials, impersonate) and Cloud KMS Autokey: a project-level escalation surface
# ArgoCD never uses. Omitting impersonate also rules out ArgoCD sync-impersonation
# (`destinationServiceAccounts` on an AppProject), which PromptQL's deployments do not set.
#
# Setting `create_argocd_custom_role = true` creates the role below and adds it to
# `ddn-automation`'s grant-restriction allowlist, so the provision can bind it to the ArgoCD
# service account instead of `roles/container.admin`. Hand `terraform output
# argocd_custom_role_name` to PromptQL so the data plane is provisioned against it.
#
# The role is `roles/container.developer` expanded, plus Kubernetes RBAC write and the
# remaining cluster-scoped object writes (admission webhook configs, legacy PSP,
# controllerRevisions, CSR approval): 416 permissions, i.e. everything in
# `container.admin` except the escalation surface described above and
# `container.clusters.connect` (see `argocd_private_endpoint_access`, below).
#
# How ArgoCD authenticates to the cluster, and why those permissions are enough: ArgoCD's
# pods mint a Google OAuth token for `argocd@...` inside *PromptQL's* project, then present
# that token as a bearer token directly to your cluster's Kubernetes API server, which
# authorizes it off the `container.*` permissions in this role. ArgoCD does not call the GKE
# control-plane API, hence no need for `clusters.getCredentials`, and the API-server
# endpoint and CA come from a stored cluster registration, not from a live GKE lookup.
locals {
argocd_custom_role_id = "promptqlDdnArgocdReconciler"
# Full resource name of the role, usable before the resource exists (count may be 0).
argocd_custom_role_name = "projects/${var.project_id}/roles/${local.argocd_custom_role_id}"
# What ddn-automation is allowed to grant to the ArgoCD service account. Both flags are
# independent: during a cutover on a live project set both (the grant-restriction condition
# gates binding *removals* too, so `container.admin` must stay allowlisted until after the
# old binding is gone). End state on a hardened project: create = true, allow = false.
argocd_grantable_roles = concat(
var.allow_container_admin_grant ? ["roles/container.admin"] : [],
var.create_argocd_custom_role ? [local.argocd_custom_role_name] : [],
)
# `container.clusters.connect` gates exactly two access paths: Connect Gateway and GKE's
# DNS-based control-plane endpoint (`*.gke.goog`). Neither is used to reach a cluster on a
# public control-plane endpoint, which authenticates on the IP endpoint plus the cluster CA
# (read via `container.clusters.get`) and the reconciler's OAuth token. It is therefore not
# in the role by default, and is opt-in via `argocd_private_endpoint_access`.
argocd_private_endpoint_permissions = [
"container.clusters.connect",
]
argocd_reconciler_base_permissions = [
"container.apiServices.create",
"container.apiServices.delete",
"container.apiServices.get",
"container.apiServices.getStatus",
"container.apiServices.list",
"container.apiServices.update",
"container.apiServices.updateStatus",
"container.auditSinks.create",
"container.auditSinks.delete",
"container.auditSinks.get",
"container.auditSinks.list",
"container.auditSinks.update",
"container.backendConfigs.create",
"container.backendConfigs.delete",
"container.backendConfigs.get",
"container.backendConfigs.list",
"container.backendConfigs.update",
"container.bindings.create",
"container.bindings.delete",
"container.bindings.get",
"container.bindings.list",
"container.bindings.update",
"container.certificateSigningRequests.approve",
"container.certificateSigningRequests.create",
"container.certificateSigningRequests.delete",
"container.certificateSigningRequests.get",
"container.certificateSigningRequests.getStatus",
"container.certificateSigningRequests.list",
"container.certificateSigningRequests.update",
"container.certificateSigningRequests.updateStatus",
"container.clusterRoleBindings.create",
"container.clusterRoleBindings.delete",
"container.clusterRoleBindings.get",
"container.clusterRoleBindings.list",
"container.clusterRoleBindings.update",
"container.clusterRoles.bind",
"container.clusterRoles.create",
"container.clusterRoles.delete",
"container.clusterRoles.escalate",
"container.clusterRoles.get",
"container.clusterRoles.list",
"container.clusterRoles.update",
"container.clusters.get",
"container.clusters.list",
"container.componentStatuses.get",
"container.componentStatuses.list",
"container.configMaps.create",
"container.configMaps.delete",
"container.configMaps.get",
"container.configMaps.list",
"container.configMaps.update",
"container.controllerRevisions.create",
"container.controllerRevisions.delete",
"container.controllerRevisions.get",
"container.controllerRevisions.list",
"container.controllerRevisions.update",
"container.cronJobs.create",
"container.cronJobs.delete",
"container.cronJobs.get",
"container.cronJobs.getStatus",
"container.cronJobs.list",
"container.cronJobs.update",
"container.cronJobs.updateStatus",
"container.csiDrivers.create",
"container.csiDrivers.delete",
"container.csiDrivers.get",
"container.csiDrivers.list",
"container.csiDrivers.update",
"container.csiNodeInfos.create",
"container.csiNodeInfos.delete",
"container.csiNodeInfos.get",
"container.csiNodeInfos.list",
"container.csiNodeInfos.update",
"container.csiNodes.create",
"container.csiNodes.delete",
"container.csiNodes.get",
"container.csiNodes.list",
"container.csiNodes.update",
"container.customResourceDefinitions.create",
"container.customResourceDefinitions.delete",
"container.customResourceDefinitions.get",
"container.customResourceDefinitions.getStatus",
"container.customResourceDefinitions.list",
"container.customResourceDefinitions.update",
"container.customResourceDefinitions.updateStatus",
"container.daemonSets.create",
"container.daemonSets.delete",
"container.daemonSets.get",
"container.daemonSets.getStatus",
"container.daemonSets.list",
"container.daemonSets.update",
"container.daemonSets.updateStatus",
"container.deployments.create",
"container.deployments.delete",
"container.deployments.get",
"container.deployments.getScale",
"container.deployments.getStatus",
"container.deployments.list",
"container.deployments.rollback",
"container.deployments.update",
"container.deployments.updateScale",
"container.deployments.updateStatus",
"container.endpointSlices.create",
"container.endpointSlices.delete",
"container.endpointSlices.get",
"container.endpointSlices.list",
"container.endpointSlices.update",
"container.endpoints.create",
"container.endpoints.delete",
"container.endpoints.get",
"container.endpoints.list",
"container.endpoints.update",
"container.events.create",
"container.events.delete",
"container.events.get",
"container.events.list",
"container.events.update",
"container.frontendConfigs.create",
"container.frontendConfigs.delete",
"container.frontendConfigs.get",
"container.frontendConfigs.list",
"container.frontendConfigs.update",
"container.horizontalPodAutoscalers.create",
"container.horizontalPodAutoscalers.delete",
"container.horizontalPodAutoscalers.get",
"container.horizontalPodAutoscalers.getStatus",
"container.horizontalPodAutoscalers.list",
"container.horizontalPodAutoscalers.update",
"container.horizontalPodAutoscalers.updateStatus",
"container.ingresses.create",
"container.ingresses.delete",
"container.ingresses.get",
"container.ingresses.getStatus",
"container.ingresses.list",
"container.ingresses.update",
"container.ingresses.updateStatus",
"container.initializerConfigurations.create",
"container.initializerConfigurations.delete",
"container.initializerConfigurations.get",
"container.initializerConfigurations.list",
"container.initializerConfigurations.update",
"container.jobs.create",
"container.jobs.delete",
"container.jobs.get",
"container.jobs.getStatus",
"container.jobs.list",
"container.jobs.update",
"container.jobs.updateStatus",
"container.leases.create",
"container.leases.delete",
"container.leases.get",
"container.leases.list",
"container.leases.update",
"container.limitRanges.create",
"container.limitRanges.delete",
"container.limitRanges.get",
"container.limitRanges.list",
"container.limitRanges.update",
"container.localSubjectAccessReviews.create",
"container.localSubjectAccessReviews.list",
"container.managedCertificates.create",
"container.managedCertificates.delete",
"container.managedCertificates.get",
"container.managedCertificates.list",
"container.managedCertificates.update",
"container.mutatingWebhookConfigurations.create",
"container.mutatingWebhookConfigurations.delete",
"container.mutatingWebhookConfigurations.get",
"container.mutatingWebhookConfigurations.list",
"container.mutatingWebhookConfigurations.update",
"container.namespaces.create",
"container.namespaces.delete",
"container.namespaces.finalize",
"container.namespaces.get",
"container.namespaces.getStatus",
"container.namespaces.list",
"container.namespaces.update",
"container.namespaces.updateStatus",
"container.networkPolicies.create",
"container.networkPolicies.delete",
"container.networkPolicies.get",
"container.networkPolicies.list",
"container.networkPolicies.update",
"container.nodes.create",
"container.nodes.delete",
"container.nodes.get",
"container.nodes.getStatus",
"container.nodes.list",
"container.nodes.proxy",
"container.nodes.update",
"container.nodes.updateStatus",
"container.persistentVolumeClaims.create",
"container.persistentVolumeClaims.delete",
"container.persistentVolumeClaims.get",
"container.persistentVolumeClaims.getStatus",
"container.persistentVolumeClaims.list",
"container.persistentVolumeClaims.update",
"container.persistentVolumeClaims.updateStatus",
"container.persistentVolumes.create",
"container.persistentVolumes.delete",
"container.persistentVolumes.get",
"container.persistentVolumes.getStatus",
"container.persistentVolumes.list",
"container.persistentVolumes.update",
"container.persistentVolumes.updateStatus",
"container.petSets.create",
"container.petSets.delete",
"container.petSets.get",
"container.petSets.list",
"container.petSets.update",
"container.petSets.updateStatus",
"container.podDisruptionBudgets.create",
"container.podDisruptionBudgets.delete",
"container.podDisruptionBudgets.get",
"container.podDisruptionBudgets.getStatus",
"container.podDisruptionBudgets.list",
"container.podDisruptionBudgets.update",
"container.podDisruptionBudgets.updateStatus",
"container.podPresets.create",
"container.podPresets.delete",
"container.podPresets.get",
"container.podPresets.list",
"container.podPresets.update",
"container.podSecurityPolicies.create",
"container.podSecurityPolicies.delete",
"container.podSecurityPolicies.get",
"container.podSecurityPolicies.list",
"container.podSecurityPolicies.update",
"container.podSecurityPolicies.use",
"container.podTemplates.create",
"container.podTemplates.delete",
"container.podTemplates.get",
"container.podTemplates.list",
"container.podTemplates.update",
"container.pods.attach",
"container.pods.create",
"container.pods.delete",
"container.pods.evict",
"container.pods.exec",
"container.pods.get",
"container.pods.getLogs",
"container.pods.getStatus",
"container.pods.initialize",
"container.pods.list",
"container.pods.portForward",
"container.pods.proxy",
"container.pods.update",
"container.pods.updateStatus",
"container.priorityClasses.create",
"container.priorityClasses.delete",
"container.priorityClasses.get",
"container.priorityClasses.list",
"container.priorityClasses.update",
"container.replicaSets.create",
"container.replicaSets.delete",
"container.replicaSets.get",
"container.replicaSets.getScale",
"container.replicaSets.getStatus",
"container.replicaSets.list",
"container.replicaSets.update",
"container.replicaSets.updateScale",
"container.replicaSets.updateStatus",
"container.replicationControllers.create",
"container.replicationControllers.delete",
"container.replicationControllers.get",
"container.replicationControllers.getScale",
"container.replicationControllers.getStatus",
"container.replicationControllers.list",
"container.replicationControllers.update",
"container.replicationControllers.updateScale",
"container.replicationControllers.updateStatus",
"container.resourceQuotas.create",
"container.resourceQuotas.delete",
"container.resourceQuotas.get",
"container.resourceQuotas.getStatus",
"container.resourceQuotas.list",
"container.resourceQuotas.update",
"container.resourceQuotas.updateStatus",
"container.roleBindings.create",
"container.roleBindings.delete",
"container.roleBindings.get",
"container.roleBindings.list",
"container.roleBindings.update",
"container.roles.bind",
"container.roles.create",
"container.roles.delete",
"container.roles.escalate",
"container.roles.get",
"container.roles.list",
"container.roles.update",
"container.runtimeClasses.create",
"container.runtimeClasses.delete",
"container.runtimeClasses.get",
"container.runtimeClasses.list",
"container.runtimeClasses.update",
"container.scheduledJobs.create",
"container.scheduledJobs.delete",
"container.scheduledJobs.get",
"container.scheduledJobs.list",
"container.scheduledJobs.update",
"container.scheduledJobs.updateStatus",
"container.secrets.create",
"container.secrets.delete",
"container.secrets.get",
"container.secrets.list",
"container.secrets.update",
"container.selfSubjectAccessReviews.create",
"container.selfSubjectAccessReviews.list",
"container.selfSubjectRulesReviews.create",
"container.serviceAccounts.create",
"container.serviceAccounts.createToken",
"container.serviceAccounts.delete",
"container.serviceAccounts.get",
"container.serviceAccounts.list",
"container.serviceAccounts.update",
"container.services.create",
"container.services.delete",
"container.services.get",
"container.services.getStatus",
"container.services.list",
"container.services.proxy",
"container.services.update",
"container.services.updateStatus",
"container.statefulSets.create",
"container.statefulSets.delete",
"container.statefulSets.get",
"container.statefulSets.getScale",
"container.statefulSets.getStatus",
"container.statefulSets.list",
"container.statefulSets.update",
"container.statefulSets.updateScale",
"container.statefulSets.updateStatus",
"container.storageClasses.create",
"container.storageClasses.delete",
"container.storageClasses.get",
"container.storageClasses.list",
"container.storageClasses.update",
"container.storageStates.create",
"container.storageStates.delete",
"container.storageStates.get",
"container.storageStates.getStatus",
"container.storageStates.list",
"container.storageStates.update",
"container.storageStates.updateStatus",
"container.storageVersionMigrations.create",
"container.storageVersionMigrations.delete",
"container.storageVersionMigrations.get",
"container.storageVersionMigrations.getStatus",
"container.storageVersionMigrations.list",
"container.storageVersionMigrations.update",
"container.storageVersionMigrations.updateStatus",
"container.subjectAccessReviews.create",
"container.subjectAccessReviews.list",
"container.thirdPartyObjects.create",
"container.thirdPartyObjects.delete",
"container.thirdPartyObjects.get",
"container.thirdPartyObjects.list",
"container.thirdPartyObjects.update",
"container.thirdPartyResources.create",
"container.thirdPartyResources.delete",
"container.thirdPartyResources.get",
"container.thirdPartyResources.list",
"container.thirdPartyResources.update",
"container.tokenReviews.create",
"container.updateInfos.create",
"container.updateInfos.delete",
"container.updateInfos.get",
"container.updateInfos.list",
"container.updateInfos.update",
"container.validatingWebhookConfigurations.create",
"container.validatingWebhookConfigurations.delete",
"container.validatingWebhookConfigurations.get",
"container.validatingWebhookConfigurations.list",
"container.validatingWebhookConfigurations.update",
"container.volumeAttachments.create",
"container.volumeAttachments.delete",
"container.volumeAttachments.get",
"container.volumeAttachments.getStatus",
"container.volumeAttachments.list",
"container.volumeAttachments.update",
"container.volumeAttachments.updateStatus",
"container.volumeSnapshotClasses.create",
"container.volumeSnapshotClasses.delete",
"container.volumeSnapshotClasses.get",
"container.volumeSnapshotClasses.list",
"container.volumeSnapshotClasses.update",
"container.volumeSnapshotContents.create",
"container.volumeSnapshotContents.delete",
"container.volumeSnapshotContents.get",
"container.volumeSnapshotContents.getStatus",
"container.volumeSnapshotContents.list",
"container.volumeSnapshotContents.update",
"container.volumeSnapshotContents.updateStatus",
"container.volumeSnapshots.create",
"container.volumeSnapshots.delete",
"container.volumeSnapshots.get",
"container.volumeSnapshots.getStatus",
"container.volumeSnapshots.list",
"container.volumeSnapshots.update",
"container.volumeSnapshots.updateStatus",
"recommender.containerDiagnosisInsights.get",
"recommender.containerDiagnosisInsights.list",
"recommender.containerDiagnosisInsights.update",
"recommender.containerDiagnosisRecommendations.get",
"recommender.containerDiagnosisRecommendations.list",
"recommender.containerDiagnosisRecommendations.update",
"recommender.locations.get",
"recommender.locations.list",
"recommender.networkAnalyzerGkeConnectivityInsights.get",
"recommender.networkAnalyzerGkeConnectivityInsights.list",
"recommender.networkAnalyzerGkeConnectivityInsights.update",
"recommender.networkAnalyzerGkeIpAddressInsights.get",
"recommender.networkAnalyzerGkeIpAddressInsights.list",
"recommender.networkAnalyzerGkeIpAddressInsights.update",
"resourcemanager.projects.get",
"resourcemanager.projects.list",
]
argocd_reconciler_permissions = sort(concat(
local.argocd_reconciler_base_permissions,
var.argocd_private_endpoint_access ? local.argocd_private_endpoint_permissions : [],
))
}
resource "google_project_iam_custom_role" "argocd_reconciler" {
count = var.create_argocd_custom_role ? 1 : 0
project = var.project_id
role_id = local.argocd_custom_role_id
title = "PromptQL DDN - ArgoCD Data Plane Reconciler"
description = "Least-privilege replacement for roles/container.admin on PromptQL's ArgoCD service account: Kubernetes RBAC and cluster-scoped object write, without GKE cluster lifecycle, getCredentials/impersonate, clusters.connect, or Cloud KMS Autokey"
permissions = local.argocd_reconciler_permissions
stage = "GA"
depends_on = [google_project_service.apis]
}
Apply it in your project:
export GCP_PROJECT_ID="your-project-id"
gcloud auth application-default login # credentials Terraform will use
terraform init
terraform apply -var project_id="${GCP_PROJECT_ID}"
Confirm all 15 roles were created, or 16 with create_argocd_custom_role = true:
gcloud iam roles list --project=${GCP_PROJECT_ID} --filter="name:promptqlDdn" --format="value(name)" | wc -l
# Should print 15, or 16 if you enabled the optional ArgoCD reconciler role below
Note
The five roles that can modify IAM policies:
promptqlDdnProjectIamAdmin,promptqlDdnServiceAccountAdmin,promptqlDdnStorageAdmin,promptqlDdnCloudkmsAdminandpromptqlDdnSecretManagerAdmin: are bound with amodifiedGrantsByRolegrant-restriction condition. It restrictsddn-automationto granting only the specific roles the provision hands out, across project, service-account, bucket, KMS-key and secret scope (seegrantable_rolesin the Terraform), so the service account cannot escalate privileges by granting any other role.
Firecracker Host fleet
The same ddn-automation service account provisions the PromptQL Firecracker (v2) sandbox Host fleet in your project: the Host subnet with its own Cloud NAT and firewall rules, the shared internal load-balancer VIP, the Host VM service account and bootstrap-trust secret, the fleet's object-store roles and bucket bindings, and the Host VM instances with their reserved internal relay addresses.
The three Firecracker roles (promptqlDdnComputeInstanceAdmin, promptqlDdnIamRoleAdmin, promptqlDdnSecretManagerAdmin), plus the firewall and internal-address permissions added to promptqlDdnComputeNetworkAdmin, are exactly what that requires:
- Host VM image. Hosts boot a shared image kept in PromptQL's own image project.
roles/compute.imageUseron that image is held by the VM-creating identity in PromptQL's project; this Terraform does not grant image access and does not touch PromptQL's project. No image is copied into your project. - Host VM identity. The automation attaches the Host service account to a VM using the existing
roles/iam.serviceAccountUserentry inpromptqlDdnServiceAccountAdmin's grant allowlist; theactAsbinding itself is created by the data-plane provisioning, not by you. - Bootstrap-trust secret.
promptqlDdnSecretManagerAdminmay write the secret and grant the Host service accountroles/secretmanager.secretAccessoron it, but carries nosecretmanager.versions.access: the automation never reads the secret's value; only the Host reads it at bootstrap. The payload is the fleet's public CA roots. - Fleet custom roles.
promptqlDdnIamRoleAdminlets the provisioning create the fleet's object-store and Broker-fence custom roles, but not grant them: those are bound through the grant-restrictedpromptqlDdnProjectIamAdmin(project scope) andpromptqlDdnStorageAdmin(bucket scope), whose allowlists name exactly thesandboxFirecracker*roles.
Optional: least-privilege ArgoCD reconciler role
By default the provision grants roles/container.admin to PromptQL's GitOps controller
(argocd@hasura-ci-397310.iam.gserviceaccount.com), because it is the only predefined GKE role
that carries Kubernetes RBAC write. It also carries GKE cluster lifecycle,
clusters.getCredentials, clusters.impersonate and Cloud KMS Autokey — none of which ArgoCD uses.
Set create_argocd_custom_role = true to create promptqlDdnArgocdReconciler instead: the same
RBAC and cluster-scoped object writes in 416 permissions, without that escalation surface. The
role is also added to ddn-automation's grant-restriction allowlist so the provision can bind it.
(A private control-plane endpoint additionally needs container.clusters.connect; set
argocd_private_endpoint_access = true. PromptQL will tell you if that applies.)
terraform apply \
-var project_id="${GCP_PROJECT_ID}" \
-var create_argocd_custom_role=true
terraform output argocd_custom_role_name
# => projects/your-project-id/roles/promptqlDdnArgocdReconciler
Give that value to PromptQL with your project ID; the provision then binds it instead of
roles/container.admin.
When to set allow_container_admin_grant = false
allow_container_admin_grant controls only what ddn-automation may grant, not what PromptQL
binds. The grant-restriction condition matches on modifiedGrantsByRole, which covers bindings a
setIamPolicy call adds, changes or removes — so removing the old container.admin binding
also requires container.admin to still be allowlisted.
| Situation | create_argocd_custom_role | allow_container_admin_grant |
|---|---|---|
| New project | true | false |
| Project PromptQL has already provisioned | true | true, then false once PromptQL confirms the data plane has been re-provisioned onto the custom role |
GCP deployment details to provide
Share the following with the PromptQL team:
- (Required) Project ID
- (Required) GCP Region
- (Optional) ArgoCD custom role
- The value of
terraform output argocd_custom_role_name, if you setcreate_argocd_custom_role = true. Without it, PromptQL provisions the data plane againstroles/container.adminas before.
- The value of
- (Optional) Preferred Availability Zones
- If you have specific zones which you'd like to use, please pass it along. Otherwise, PromptQL will assign accordingly.
- (Optional) VPC CIDR (/16-/19 CIDR)
- If you have a specific CIDR in mind for the VPC setup, please pass it along. If not specified, PromptQL will assign 10.0.0.0/16.
- Note: If you are planning to use VPC Peering, this CIDR should not conflict with any networks on your side.
Azure
Azure prerequisites
- Confirm the deployment model and target Azure region with the PromptQL team.
- Record the Data Plane name, domain, availability zones, VNet CIDR, and private-connectivity requirements.
- BYOC requires a dedicated Azure subscription or approved resource groups, permission to register Enterprise applications, and permission to assign roles.
- Confirm the selected region and subscription have sufficient quotas for the required services.
Azure network and availability plan
- Select one Azure region and the required availability zones.
- Reserve a non-overlapping VNet CIDR, normally /16 to /20.
- Confirm the network path to each data source: VNet peering, VPN, Azure Private Link/Private Endpoint, or another approved private path.
- For BYOC, use a dedicated subscription or approved resource groups and confirm policies and quotas permit AKS, database, storage, load-balancing, identity, and networking resources.
Azure customer tasks
- Prepare a dedicated Azure subscription or approved resource groups and confirm the target region, quotas, policies, and VNet range.
- Register the PromptQL infrastructure-management service principal in your Microsoft Entra ID tenant.
- Deploy the role-assignment template below to the approved resource groups.
- Register the required AKS API server VNet integration feature.
- Provide the subscription ID, tenant ID, resource group names, region, and network plan to PromptQL.
- Keep the service-principal registration and role assignments in place while PromptQL operates the Data Plane.
PromptQL then uses the scoped service principal to provision, upgrade, and operate the approved resources in your Azure subscription.
Detailed Azure setup
Prerequisites
- Azure Account with administrative access
- Dedicated Azure Subscription where PromptQL will provision BYOC infrastructure
- Permissions to register Enterprise applications in your Azure AD tenant
- Permissions to assign roles at the subscription level
Provisioning Access
You need to register the PromptQL provisioning service principal in your Azure AD tenant:
For Infrastructure Provisioning (PrivateDdnInfra):
- Log in with an account that has permissions to register Enterprise applications
- Open a browser tab and navigate to the following URL, replacing
<TENANT_ID>with your Azure Tenant ID:https://login.microsoftonline.com/<TENANT_ID>/oauth2/authorize?client_id=4f7f1f59-f0b0-4adb-8603-2afacc50552b&response_type=code&redirect_uri=https://hasura.io/docs/3.0/private-ddn/creating-a-data-plane/byoc - Verify the service principal details match exactly:
- Client ID:
4f7f1f59-f0b0-4adb-8603-2afacc50552b - Name:
Hasura Cloud Infrastructure Manager
- Client ID:
- Select "Consent on behalf of your organization" and click "Accept"
- You will be redirected back to this documentation page, confirming the registration was completed
After completing the registration process, you can verify that the service principal was created successfully:
- Go to the Azure Portal
- Navigate to Azure Active Directory → Enterprise applications
- Search for "Hasura Cloud Infrastructure Manager" to confirm it appears in your tenant
Use the provided Azure Bicep template to automatically set up the required roles and permissions:
- Copy the following ARM template and save it as
role.json:
role.json
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"metadata": {
"_generator": {
"name": "bicep",
"version": "0.37.4.10188",
"templateHash": "17100090430408911702"
}
},
"parameters": {
"resourceGroupNames": {
"type": "array",
"metadata": {
"description": "An array of resource group names to assign the role in."
}
},
"principalId": {
"type": "string",
"metadata": {
"description": "The principal (service principal) ID to assign the role to."
}
},
"principalType": {
"type": "string",
"defaultValue": "ServicePrincipal",
"allowedValues": [
"User",
"Group",
"ServicePrincipal"
],
"metadata": {
"description": "The type of principal (User, Group, or ServicePrincipal)."
}
}
},
"resources": [
{
"type": "Microsoft.Authorization/roleDefinitions",
"apiVersion": "2022-04-01",
"name": "[guid(subscription().id, 'HasuraCloudBYOCRole')]",
"properties": {
"roleName": "HasuraCloudBYOC",
"description": "Role for PromptQL BYOC",
"type": "CustomRole",
"assignableScopes": [
"[subscription().id]"
],
"permissions": [
{
"actions": [
"Microsoft.DBforPostgreSQL/flexibleServers/*"
]
}
]
}
},
{
"copy": {
"name": "roleAssignments",
"count": "[length(parameters('resourceGroupNames'))]"
},
"type": "Microsoft.Resources/deployments",
"apiVersion": "2022-09-01",
"name": "[format('roleAssignment-{0}', parameters('resourceGroupNames')[copyIndex()])]",
"resourceGroup": "[parameters('resourceGroupNames')[copyIndex()]]",
"properties": {
"expressionEvaluationOptions": {
"scope": "inner"
},
"mode": "Incremental",
"parameters": {
"customRoleDefinitionId": {
"value": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', guid(subscription().id, 'HasuraCloudBYOCRole'))]"
},
"principalId": {
"value": "[parameters('principalId')]"
},
"principalType": {
"value": "[parameters('principalType')]"
}
},
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"metadata": {
"_generator": {
"name": "bicep",
"version": "0.37.4.10188",
"templateHash": "4955557183172246065"
}
},
"parameters": {
"customRoleDefinitionId": {
"type": "string",
"metadata": {
"description": "The custom role definition ID to assign."
}
},
"principalId": {
"type": "string",
"metadata": {
"description": "The principal (user, group, service principal) object ID to assign the role to."
}
},
"principalType": {
"type": "string",
"defaultValue": "ServicePrincipal",
"allowedValues": [
"User",
"Group",
"ServicePrincipal"
],
"metadata": {
"description": "The type of principal (User, Group, or ServicePrincipal)."
}
}
},
"variables": {
"networkContributorRoleId": "4d97b98b-1d4f-4787-a291-c67834d212e7",
"managedIdentityContributorRoleId": "e40ec5ca-96e0-45a2-b4ff-59039f2c2b59",
"managedIdentityOperatorRoleId": "f1a07417-d97a-45cb-824c-7a7467783830",
"aksContributorRoleId": "ed7f3fbd-7b88-4dd4-9017-9adb7ce333f8",
"storageAccountContributorRoleId": "17d1049b-9a84-46fb-8f53-869881c3d3ab",
"rbacAdministratorRoleId": "f58310d9-a9f6-439a-9e8d-f62e7b41a168",
"rbacCondition": "!(ActionMatches{'Microsoft.Authorization/roleAssignments/write'} OR ActionMatches{'Microsoft.Authorization/roleAssignments/delete'}) OR (@Request[Microsoft.Authorization/roleAssignments:RoleDefinitionId] ForAnyOfAnyValues:GuidEquals {4d97b98b-1d4f-4787-a291-c67834d212e7, acdd72a7-3385-48ef-bd42-f606fba81ae7, b1ff04bb-8a4e-4dc4-8eb5-8693973ce19b, ba92f5b4-2d11-453d-a403-e96b0029c9fe, 2a2b9908-6ea1-4ae2-8e65-a410df84e7d1} AND @Request[Microsoft.Authorization/roleAssignments:PrincipalType] ForAnyOfAnyValues:StringEqualsIgnoreCase {'ServicePrincipal', 'MSI'})"
},
"resources": [
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2022-04-01",
"name": "[guid(resourceGroup().id, parameters('customRoleDefinitionId'), parameters('principalId'), 'custom')]",
"properties": {
"roleDefinitionId": "[parameters('customRoleDefinitionId')]",
"principalId": "[parameters('principalId')]",
"principalType": "[parameters('principalType')]"
}
},
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2022-04-01",
"name": "[guid(resourceGroup().id, variables('networkContributorRoleId'), parameters('principalId'), 'network')]",
"properties": {
"roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('networkContributorRoleId'))]",
"principalId": "[parameters('principalId')]",
"principalType": "[parameters('principalType')]"
}
},
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2022-04-01",
"name": "[guid(resourceGroup().id, variables('managedIdentityContributorRoleId'), parameters('principalId'), 'identity')]",
"properties": {
"roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('managedIdentityContributorRoleId'))]",
"principalId": "[parameters('principalId')]",
"principalType": "[parameters('principalType')]"
}
},
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2022-04-01",
"name": "[guid(resourceGroup().id, variables('managedIdentityOperatorRoleId'), parameters('principalId'), 'operator')]",
"properties": {
"roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('managedIdentityOperatorRoleId'))]",
"principalId": "[parameters('principalId')]",
"principalType": "[parameters('principalType')]"
}
},
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2022-04-01",
"name": "[guid(resourceGroup().id, variables('aksContributorRoleId'), parameters('principalId'), 'aks')]",
"properties": {
"roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('aksContributorRoleId'))]",
"principalId": "[parameters('principalId')]",
"principalType": "[parameters('principalType')]"
}
},
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2022-04-01",
"name": "[guid(resourceGroup().id, variables('storageAccountContributorRoleId'), parameters('principalId'), 'storage')]",
"properties": {
"roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('storageAccountContributorRoleId'))]",
"principalId": "[parameters('principalId')]",
"principalType": "[parameters('principalType')]"
}
},
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2022-04-01",
"name": "[guid(resourceGroup().id, variables('rbacAdministratorRoleId'), parameters('principalId'), 'rbac')]",
"properties": {
"roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('rbacAdministratorRoleId'))]",
"principalId": "[parameters('principalId')]",
"principalType": "[parameters('principalType')]",
"condition": "[variables('rbacCondition')]",
"conditionVersion": "2.0"
}
}
]
}
},
"dependsOn": [
"[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', guid(subscription().id, 'HasuraCloudBYOCRole'))]"
]
}
]
}
-
Deploy the template using Azure CLI:
# Set your subscription az account set --subscription "your-subscription-id" # Get the Object ID of the PromptQL provisioning service principal PRINCIPAL_ID=$(az ad sp list --filter "appId eq '4f7f1f59-f0b0-4adb-8603-2afacc50552b'" --query "[0].id" --output tsv) # Deploy the template az deployment sub create \ --name "hasura-cloud-byoc" \ --location "East US" \ --template-file role.json \ --parameters \ resourceGroupNames='["your-resource-group-name"]' \ principalId="$PRINCIPAL_ID" \ principalType="ServicePrincipal"Replace the following values:
your-subscription-id: Your Azure subscription IDyour-resource-group-name: The resource group where PromptQL will operate (you can specify multiple resource groups:'["rg1","rg2","rg3"]')"East US": Your preferred Azure region
-
Monitor the deployment:
az deployment sub show --name hasura-cloud-byoc --query 'properties.provisioningState'
Additional Requirements
Register the required Azure feature flag for API server VNet integration:
az feature register --namespace "Microsoft.ContainerService" --name "EnableAPIServerVnetIntegrationPreview"
You can check the registration status with:
az feature show --namespace "Microsoft.ContainerService" --name "EnableAPIServerVnetIntegrationPreview"
Azure deployment details to provide
Gather the following information from your Azure Portal and provide it to the PromptQL team:
Required:
- Subscription ID: The target subscription where BYOC infrastructure will be deployed
- Tenant ID: Your Azure AD tenant identifier
- Resource Group Name(s): The resource group(s) where PromptQL will operate
- Azure Region: The target region for deployment (e.g., "East US", "West Europe")
Optional:
- Preferred Availability Zones: Specific logical zone IDs within your chosen region
- If not specified, PromptQL will select zones automatically
- Use logical zone IDs (e.g., "1", "2", "3") which map to physical zones
- You can get the zone mappings by running:
Replaceaz rest --method get \ --uri '/subscriptions/{subscriptionId}/locations?api-version=2022-12-01' \ --query 'value[?name==`eastus` && availabilityZoneMappings != `null`].{displayName: displayName, name: name, availabilityZoneMappings: availabilityZoneMappings}'eastuswith your desired region name and{subscriptionId}with your subscription ID - Example: ["1", "2", "3"] for logical zones 1, 2, and 3
- VNet CIDR: Custom CIDR block for the virtual network (/16-/19 range)
- Default: 10.0.0.0/16 if not specified
- Important: Ensure this doesn't conflict with existing networks if using VNet peering