Google Cloud IAP Role Codification
By Drew Rothstein, Head of Engineering
IAP Codification in Terraform on GCP
IAP (Identity-Aware Proxy) on GCP (Google Cloud Platform) is a fantastic zero-trust service / implementation offered by Google (ref).
When used extensively it can allow you to create an easy-to-maintain and validate AuthN (and AuthZ, depending on configuration) proxy to your services.
Also - no more VPNs! Welcome to the future.
Recently, we at Labs started to codify much of our IAP role configuration in Terraform. Prior to this, our codification of IAP roles specifically was a bit spotty and during a recent infrastructure migration we found several pockets of uncodified IAP role configuration.
This post aims to show our process that we used to 1) identify and 2) codify our IAP role footprint. This was written because when we went to find a similar resource, we couldn’t find one and felt it necessary and appropriate to share. This post took more time to write than the work contained within but this can hopefully serve as a valuable resource for others that embark on this similar journey in the future. Due to this one-time operation and the iterative nature required shell scripting was chosen.
Identify
IAP can be configured with an Organization (herein “org”), a Folder, on a Project, on Compute, Web, and App Engine resources. Using the gcloud CLI (ref) you can easily start to put together your current IAP footprint.
Note: You will need / want to perform the below on each org that you manage.
To view settings on your org:
ORG_ID=<111222233333>
gcloud iap settings get --organization $ORG_ID
This may print an error that you do not have permission or there are no settings if no settings are present at the org level.
To view settings on each folder in your org:
ORG_ID=<>
for folder in $(gcloud resource-manager folders list --organization "$ORG_ID" | awk 'NR>1 {print $3}');
do
gcloud iap settings get --folder "$folder";
done
This may print the id of the folders with no output (if not set at the folder-level).
To view that settings exist on each project in your org (this does not list the actual settings):
projects_iap=();
for project in $(gcloud projects list | grep -v sys | awk 'NR>1 {print $1}');
do
echo "Checking IAP in Project: ${project} ...";
if [[ $(gcloud iap settings get --project "$project" --quiet 2> /dev/null) ]];
then
projects_iap+=("$project");
fi
done
echo "${projects_iap[@]}";
This will print that is checking each project and then at the end provide a list of projects with IAP settings. This will swallow stdout as to not print about projects where IAP is not enabled.
Backend Service Identification
If you run backend services that utilize IAP you may be wondering how to actually view the IAM membership of IAP-Secured Web App User bindings since that is what is actually used to authorize access to your service. This output is not included with gcloud iap settings get and is a separate CLI / API call to view.
We begin our journey with gcloud alpha. The specific command we are interested in is gcloud alpha iap web get-iam-policy which will allow us to view the IAM policy on a specific backend resource.
Note: This command specifies resource-type as either app-engine or backend-services which is different from the previous command which accepts app-engine, iap_web, compute, organization, or folder.
To be able to view the bindings over multiple projects you will need to specify the project, service and resource-type.
Taking the output of the projects above, we can iterate the backend services and parse / produce the output we seek.
Note: This enables the cloudresourcemanager on the projects if it is not enabled. You will not get valid output (the IAM policies) if this API is not enabled. You will need the serviceusage.services.enable permission for your user to be able to enable.
for project in "${projects_iap[@]}";
do
echo "Checking Backends in ${project} ...";
backends_configured=();
backends=$(gcloud compute backend-services list --project "$project" --quiet 2> /dev/null | awk 'NR>1 {print $1}')
for backend in $backends;
do
backends_configured+=("$backend");
done
echo "Enabling CRM API on ${project} ...";
gcloud services enable cloudresourcemanager.googleapis.com --project "$project"
for backend_service in "${backends_configured[@]}";
do
echo "IAM Policy for $backend_service ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --service "$backend_service" --resource-type backend-services --quiet 2> /dev/null)
if [[ $iam_policy ]];
then
echo "${iam_policy}";
else
echo ">> No IAM Policy Configured."
fi
done
done
With IAP and App Engine you need to iterate the top-level App and the individual services as they may have different settings. This is slightly different than with backend services. Most of the time you will likely set the top-level and the services will inherit.
If you utilize App Engine w/IAP, you can also check those services similarly with:
for project in "${projects_iap[@]}";
do
echo "Checking App Engine services in ${project} ...";
services_configured=();
services=$(gcloud app services list --project "$project" --quiet 2> /dev/null | awk 'NR>1 {print $1}')
for service in $services;
do
services_configured+=("$service");
done
# Assuming CRM was enabled previously
# echo "Enabling CRM API on ${project} ...";
# gcloud services enable cloudresourcemanager.googleapis.com --project "$project"
echo "IAM Policy for top-level App Engine App ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --resource-type app-engine --quiet 2> /dev/null | grep -v -e "version" -e "etag")
if [[ $iam_policy ]];
then
echo "${iam_policy}";
else
echo ">> No IAM Policy Configured."
fi
for appengine_service in "${services_configured[@]}";
do
echo "IAM Policy for App Engine service: $appengine_service ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --resource-type app-engine --service "$appengine_service" --quiet 2> /dev/null | grep -v -e "version" -e "etag")
if [[ $iam_policy ]];
then
echo "${iam_policy}";
else
echo ">> No IAM Policy Configured."
fi
done
done
This is great- we can now view all our current membership. A common event prior to codification is to update IAM or fix / remediate settings that were potentially done manually but not what you are seeking to codify.
For example, you may want to modify all IAM role policies for IAP-secured backend services where a specific member is present. In the following you could provide a search term to remove and a member to add if found. Our use case was removing human (user prefix) and replacing it with a group syntax.
REMOVE_MEMBER="user:[email protected]"
ADD_MEMBER="group:[email protected]"
IAP_ROLE="roles/iap.httpsResourceAccessor"
for project in "${projects_iap[@]}";
do
echo "Checking Backends in ${project} ...";
backends_configured=();
backends=$(gcloud compute backend-services list --project "$project" --quiet 2> /dev/null | awk 'NR>1 {print $1}')
for backend in $backends;
do
backends_configured+=("$backend");
done
# Assuming CRM was enabled previously
# echo "Enabling CRM API on ${project} ...";
# gcloud services enable cloudresourcemanager.googleapis.com --project "$project"
for backend_service in "${backends_configured[@]}";
do
echo "IAM Policy for $backend_service ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --service "$backend_service" --resource-type backend-services --quiet 2> /dev/null)
if [[ $iam_policy ]];
then
if echo "${iam_policy}" | grep -q "${REMOVE_MEMBER}";
then
echo "Found match for ${REMOVE_MEMBER} ..."
echo "Adding member ${ADD_MEMBER} to policy for service ${backend_service} ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web add-iam-policy-binding --project "$project" --service "$backend_service" --member "$ADD_MEMBER" --role "$IAP_ROLE" --resource-type backend-services
fi
echo "Removing member ${REMOVE_MEMBER} from policy for service ${backend_service} ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web remove-iam-policy-binding --project "$project" --service "$backend_service" --member "$REMOVE_MEMBER" --role "$IAP_ROLE" --resource-type backend-services
fi
fi
else
echo ">> No IAM Policy Configured."
fi
done
done
Similarly with App Engine, to remove a specific member and add another:
REMOVE_MEMBER="user:[email protected]"
ADD_MEMBER="group:[email protected]"
IAP_ROLE="roles/iap.httpsResourceAccessor"
for project in "${projects_iap[@]}";
do
echo "Checking App Engine services in ${project} ...";
services_configured=();
services=$(gcloud app services list --project "$project" --quiet 2> /dev/null | awk 'NR>1 {print $1}')
for service in $services;
do
services_configured+=("$service");
done
# Assuming CRM was enabled previously
# echo "Enabling CRM API on ${project} ...";
# gcloud services enable cloudresourcemanager.googleapis.com --project "$project"
echo "IAM Policy for top-level App Engine App ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --resource-type app-engine --quiet 2> /dev/null | grep -v -e "version" -e "etag")
if [[ $iam_policy ]];
then
if echo "${iam_policy}" | grep -q "${REMOVE_MEMBER}";
then
echo "Found match for ${REMOVE_MEMBER} ..."
echo "Adding member ${ADD_MEMBER} to policy for top-level App Engine App ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web add-iam-policy-binding --project "$project" --member "$ADD_MEMBER" --role "$IAP_ROLE" --resource-type app-engine
fi
echo "Removing member ${REMOVE_MEMBER} from policy for top-level App Engine App ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web remove-iam-policy-binding --project "$project" --member "$REMOVE_MEMBER" --role "$IAP_ROLE" --resource-type app-engine
fi
fi
else
echo ">> No IAM Policy Configured."
fi
for appengine_service in "${services_configured[@]}";
do
echo "IAM Policy for App Engine service: $appengine_service ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --resource-type app-engine --service "$appengine_service" --quiet 2> /dev/null | grep -v -e "version" -e "etag")
if [[ $iam_policy ]]; then
if echo "${iam_policy}" | grep -q "${REMOVE_MEMBER}";
then
echo "Found match for ${REMOVE_MEMBER} ..."
echo "Adding member ${ADD_MEMBER} to policy for service ${appengine_service} ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web add-iam-policy-binding --project "$project" --service "$appengine_service" --member "$ADD_MEMBER" --role "$IAP_ROLE" --resource-type app-engine
fi
echo "Removing member ${REMOVE_MEMBER} from policy for service ${appengine_service} ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web remove-iam-policy-binding --project "$project" --service "$appengine_service" --member "$REMOVE_MEMBER" --role "$IAP_ROLE" --resource-type app-engine
fi
fi
else
echo ">> No IAM Policy Configured."
fi
done
done
To iterate and remove a specified member, this can be a bit shortened:
REMOVE_MEMBER="user:[email protected]"
IAP_ROLE="roles/iap.httpsResourceAccessor"
for project in "${projects_iap[@]}";
do
echo "Checking Backends in ${project} ...";
backends_configured=();
backends=$(gcloud compute backend-services list --project "$project" --quiet 2> /dev/null | awk 'NR>1 {print $1}')
for backend in $backends;
do
backends_configured+=("$backend");
done
# Assuming CRM was enabled previously
# echo "Enabling CRM API on ${project} ...";
# gcloud services enable cloudresourcemanager.googleapis.com --project "$project"
for backend_service in "${backends_configured[@]}";
do
echo "IAM Policy for $backend_service ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --service "$backend_service" --resource-type backend-services --quiet 2> /dev/null)
if [[ $iam_policy ]];
then
if echo "${iam_policy}" | grep -q "${REMOVE_MEMBER}";
then
echo "Found match for ${REMOVE_MEMBER} ..."
echo "Removing member ${REMOVE_MEMBER} from policy for service ${backend_service} ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web remove-iam-policy-binding --project "$project" --service "$backend_service" --member "$REMOVE_MEMBER" --role "$IAP_ROLE" --resource-type backend-services
fi
fi
else
echo ">> No IAM Policy Configured."
fi
done
done
Similarly with App Engine, to remove a specific member:
REMOVE_MEMBER="user:[email protected]"
IAP_ROLE="roles/iap.httpsResourceAccessor"
for project in "${projects_iap[@]}";
do
echo "Checking App Engine services in ${project} ...";
services_configured=();
services=$(gcloud app services list --project "$project" --quiet 2> /dev/null | awk 'NR>1 {print $1}')
for service in $services;
do
services_configured+=("$service");
done
# Assuming CRM was enabled previously
# echo "Enabling CRM API on ${project} ...";
# gcloud services enable cloudresourcemanager.googleapis.com --project "$project"
echo "IAM Policy for top-level App Engine App ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --resource-type app-engine --quiet 2> /dev/null | grep -v -e "version" -e "etag")
if [[ $iam_policy ]];
then
if echo "${iam_policy}" | grep -q "${REMOVE_MEMBER}";
then
echo "Found match for ${REMOVE_MEMBER} ..."
echo "Removing member ${REMOVE_MEMBER} from policy for top-level App Engine App ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web remove-iam-policy-binding --project "$project" --member "$REMOVE_MEMBER" --role "$IAP_ROLE" --resource-type app-engine
fi
fi
else
echo ">> No IAM Policy Configured."
fi
for appengine_service in "${services_configured[@]}";
do
echo "IAM Policy for App Engine service: $appengine_service ..."
iam_policy=$(gcloud alpha iap web get-iam-policy --project "$project" --resource-type app-engine --service "$appengine_service" --quiet 2> /dev/null | grep -v -e "version" -e "etag")
if [[ $iam_policy ]];
then
if echo "${iam_policy}" | grep -q "${REMOVE_MEMBER}";
then
echo "Found match for ${REMOVE_MEMBER} ..."
echo "Removing member ${REMOVE_MEMBER} from policy for service ${appengine_service} ..."
read -p "Are you sure? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]];
then
gcloud alpha iap web remove-iam-policy-binding --project "$project" --service "$appengine_service" --member "$REMOVE_MEMBER" --role "$IAP_ROLE" --resource-type app-engine
fi
fi
else
echo ">> No IAM Policy Configured."
fi
done
done
Assuming your IAM policies are now updated in the way you would like them, we now move on to codification of these policies in Terraform.
Codify
Following identification, you now have a list of projects that have resources with IAP roles configured which is fantastic. You now understand the state of the world and can begin codification of these resources.
The next part of this section is highly dependent on how you codify in Terraform today. At Labs, we use separate state files for each project and each project is scoped to a single service. This may not be the case for you with a single larger state file (fairly common across the industry). As an aside, if you haven’t already started to break that down into separate pieces per service - now might be the time to start that journey or at least bring it up with your team (similar to not running single large clusters for diverse workloads).
Codification via Terraform for GCP’s IAP is unnecessarily complex. It appears to be auto-generated given the complex naming and formatting. Julia Evan’s post on GCP IAM is spot-on (ref) on this topic and leads me to this conclusion / confirmation.
The boiler-plate Terraform isn’t super relevant to this post but we do something roughly equivalent to this:
terraform {
backend "gcs" {
bucket = "tf-state-BUCKET-NAME"
}
}
provider "google" {
project = "SOME-PROJECT"
region = "us-west1"
}
provider "google-beta" {
project = "SOME-PROJECT"
region = "us-west1"
}
An example for App Engine services:
# Docs: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/iap_web_type_app_engine_iam
resource "google_iap_web_type_app_engine_iam_binding" "PROJECT-appengine" {
project = "PROJECT"
app_id = "PROJECT"
role = "roles/iap.httpsResourceAccessor"
members = [
"domain:SOME-DOMAIN",
"group:SOME-GROUP",
]
}
Once defined, an import will look like this:
terraform import google_iap_web_type_app_engine_iam_binding.PROJECT-appengine "projects/PROJECT/iap_web/appengine-PROJECT/services/default roles/iap.httpsResourceAccessor"
An example for Backend services:
# Doc: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/iap_web_backend_service_iam#google_iap_web_backend_service_iam_binding
resource "google_iap_web_backend_service_iam_binding" "PROJECT-backend" {
project = "PROJECT"
web_backend_service = "SERVICE"
role = "roles/iap.httpsResourceAccessor"
members = [
"group:SOME-GROUP",
"serviceAccount:SOME-SA",
]
}
Once defined, an import will look like this:
terraform import google_iap_web_backend_service_iam_binding.PROJECT-backend "projects/PROJECT/iap_web/compute/services/BACKEND-SERVICE roles/iap.httpsResourceAccessor"
We chose to use binding vs. policy and member syntax. The reason for this choice was that it was authoritative for the given role which was relevant to how we define / codify our services. It may be more logical for you to codify with policy and / or member - just note the “Note”’s in the Terraform documentation if you choose this route.
Using the small script snippets above we were able to iterate and confirm that each usage of IAP roles is now codified! Our team reviewed and discussed various improvements we would want to make in the future but the 0 to 80% of going from fairly uncodified to codified for our IAP role footprint was achieved.
Conclusion
GCP’s IAP service is not terribly complex to codify but does require a bit of scripting with alpha CLI commands / APIs to accomplish at this time. In addition, while their documentation is a bit sparse (and naming is barely differentiated in Terraform) this is not too difficult once you know where to look and how to parse the naming.
Codification will be organization and team dependent but hopefully this serves as a searchable resource for others that encounter a similar challenge in the future.
This is a short example of an afternoon project on the Infrastructure side of our Engineering team here at Labs. If you are interested in this type of work, feel free to get in touch with us via dev [at] unit410 [dot] com.