Upload objects to Google Cloud Storage using Terraform

Introduction

I will describe Cloud Storage of Google Cloud Platform (GCP) using Terraform.
(Applicable to this article)
– Interested in managing GCP using Terraform
(Content of verification)
– Run Terraform to create Cloud Storage buckets.
– Upload the following objects to the bucket.
— image
— Text
(Terraform execution environment)
– Virtualization software: Oracle VM VirtualBox 5.2.4
– OS: CentOS 7.4
– Terraform: v0.11.2
– Internet connection: possible
(Test target environment)
– Cloud environment: Google Compute Engine
– Service: Google Cloud Storage
(Reference information)
URL: Terraform – google_storage_bucket_object

1.Create buckets for Google Cloud Storage

1.1.Create a terraform tf file.

# cat gcs.tf
variable "cpath" { }
variable "project" { }
variable "region" { }

provider "google" {
  credentials  = "${file("${var.cpath}")}"
  project      = "${var.project}"
  region       = "${var.region}"
}

resource "google_storage_bucket" "COLDLINE" {
  name     = "curry999904"
  storage_class = "COLDLINE"
  location = "asia-northeast1"
}

resource "google_storage_bucket_object" "picture" {
  name   = "butterfly01"
  source = "sample.eps"
  bucket = "${google_storage_bucket.COLDLINE.name}"
}

resource "google_storage_bucket_object" "text" {
  name   = "butterfly02"
  content = "aaabbbccc"
  bucket = "${google_storage_bucket.COLDLINE.name}"
}

1.2.Create terraform environment variable file.

# cat config.tfvars
cpath="/var/credentials/account.json"
project="projcet-crossbar-999999"
region="asia-northeast1"

1.3.Initialize Terraform.

# terraform init
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "google" (1.4.0)...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.google: version = "~> 1.4"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

1.4.Plan for Terraform.

# terraform plan -out=tfplan -var-file=config.tfvars
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + google_storage_bucket.COLDLINE
      id:             
      force_destroy:  "false"
      location:       "ASIA-NORTHEAST1"
      name:           "curry999904"
      project:        
      self_link:      
      storage_class:  "COLDLINE"
      url:            

  + google_storage_bucket_object.picture
      id:             
      bucket:         "curry999904"
      content_type:   
      crc32c:         
      detect_md5hash: "different hash"
      md5hash:        
      name:           "butterfly01"
      source:         "sample.eps"
      storage_class:  

  + google_storage_bucket_object.text
      id:             
      bucket:         "curry999904"
      content:        "aaabbbccc"
      content_type:   
      crc32c:         
      detect_md5hash: "different hash"
      md5hash:        
      name:           "butterfly02"
      storage_class:  


Plan: 3 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

This plan was saved to: tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan"

1.5.Run Terraform.

# terraform apply "tfplan"
google_storage_bucket.COLDLINE: Creating...
  force_destroy: "" => "false"
  location:      "" => "ASIA-NORTHEAST1"
  name:          "" => "curry999904"
  project:       "" => ""
  self_link:     "" => ""
  storage_class: "" => "COLDLINE"
  url:           "" => ""
google_storage_bucket.COLDLINE: Creation complete after 4s (ID: curry999904)
google_storage_bucket_object.text: Creating...
  bucket:         "" => "curry999904"
  content:        "" => "aaabbbccc"
  content_type:   "" => ""
  crc32c:         "" => ""
  detect_md5hash: "" => "different hash"
  md5hash:        "" => ""
  name:           "" => "butterfly02"
  storage_class:  "" => ""
google_storage_bucket_object.picture: Creating...
  bucket:         "" => "curry999904"
  content_type:   "" => ""
  crc32c:         "" => ""
  detect_md5hash: "" => "different hash"
  md5hash:        "" => ""
  name:           "" => "butterfly01"
  source:         "" => "sample.eps"
  storage_class:  "" => ""
google_storage_bucket_object.text: Creation complete after 1s (ID: curry999904-butterfly02)
google_storage_bucket_object.picture: Creation complete after 2s (ID: curry999904-butterfly01)

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

1.6.Check the status of terraform.

# terraform show
google_storage_bucket.COLDLINE:
  id = curry999904
  cors.# = 0
  force_destroy = false
  labels.% = 0
  location = ASIA-NORTHEAST1
  name = curry999904
  project = 
  self_link = 
  storage_class = COLDLINE
  url = gs://curry999904
  versioning.# = 0
google_storage_bucket_object.picture:
  id = curry999904-butterfly01
  bucket = curry999904
  cache_control =
  content_disposition =
  content_encoding =
  content_language =
  content_type = application/octet-stream
  crc32c = NzUi7w==
  detect_md5hash = 3qWaTCGyUkCjOUnwQlTIJg==
  md5hash = 3qWaTCGyUkCjOUnwQlTIJg==
  name = butterfly01
  source = sample.eps
  storage_class = COLDLINE
google_storage_bucket_object.text:
  id = curry999904-butterfly02
  bucket = curry999904
  cache_control =
  content = aaabbbccc
  content_disposition =
  content_encoding =
  content_language =
  content_type = text/plain; charset=utf-8
  crc32c = wHwIBQ==
  detect_md5hash = 0ar0dno8EKRzQHpOR7Atpg==
  md5hash = 0ar0dno8EKRzQHpOR7Atpg==
  name = butterfly02
  storage_class = COLDLINE

2.Change Google Cloud Storage bucket

2.1.Correct the terraform tf file.

# cat gcs.tf
variable "cpath" { }
variable "project" { }
variable "region" { }

provider "google" {
  credentials  = "${file("${var.cpath}")}"
  project      = "${var.project}"
  region       = "${var.region}"
}

resource "google_storage_bucket" "COLDLINE" {
  name     = "curry999904"
  storage_class = "COLDLINE"
  location = "asia-northeast1"
}

resource "google_storage_bucket_object" "picture" {
  name   = "butterfly01"
  source = "sample.eps"
  bucket = "${google_storage_bucket.COLDLINE.name}"
}

resource "google_storage_bucket_object" "text" {
  name   = "butterfly02"
  content = "dddddddddddddddddd"
  bucket = "${google_storage_bucket.COLDLINE.name}"
}

2.2.Create terraform environment variable file.

# cat config.tfvars
cpath="/var/credentials/account.json"
project="projcet-crossbar-999999"
region="asia-northeast1"

2.3.Initialize Terraform.

# terraform init
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "google" (1.4.0)...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.google: version = "~> 1.4"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

2.4.Plan for Terraform.

# terraform plan -out=tfplan -var-file=config.tfvars
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

google_storage_bucket.COLDLINE: Refreshing state... (ID: curry999904)
google_storage_bucket_object.picture: Refreshing state... (ID: curry999904-butterfly01)
google_storage_bucket_object.text: Refreshing state... (ID: curry999904-butterfly02)

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

-/+ google_storage_bucket_object.text (new resource required)
      id:             "curry999904-butterfly02" =>  (forces new resource)
      bucket:         "curry999904" => "curry999904"
      content:        "aaabbbccc" => "dddddddddddddddddd" (forces new resource)
      content_type:   "text/plain; charset=utf-8" => 
      crc32c:         "wHwIBQ==" => 
      detect_md5hash: "0ar0dno8EKRzQHpOR7Atpg==" => "different hash" (forces new resource)
      md5hash:        "0ar0dno8EKRzQHpOR7Atpg==" => 
      name:           "butterfly02" => "butterfly02"
      storage_class:  "COLDLINE" => 


Plan: 1 to add, 0 to change, 1 to destroy.

------------------------------------------------------------------------

This plan was saved to: tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan"

2.5.Run Terraform.

# terraform apply "tfplan"
google_storage_bucket_object.text: Destroying... (ID: curry999904-butterfly02)
google_storage_bucket_object.text: Destruction complete after 0s
google_storage_bucket_object.text: Creating...
  bucket:         "" => "curry999904"
  content:        "" => "dddddddddddddddddd"
  content_type:   "" => ""
  crc32c:         "" => ""
  detect_md5hash: "" => "different hash"
  md5hash:        "" => ""
  name:           "" => "butterfly02"
  storage_class:  "" => ""
google_storage_bucket_object.text: Creation complete after 1s (ID: curry999904-butterfly02)

Apply complete! Resources: 1 added, 0 changed, 1 destroyed.

2.6.Check the status of terraform.

# terraform show
google_storage_bucket.COLDLINE:
  id = curry999904
  cors.# = 0
  force_destroy = false
  labels.% = 0
  location = ASIA-NORTHEAST1
  name = curry999904
  project = 
  self_link = 
  storage_class = COLDLINE
  url = gs://curry999904
  versioning.# = 0
google_storage_bucket_object.picture:
  id = curry999904-butterfly01
  bucket = curry999904
  cache_control =
  content_disposition =
  content_encoding =
  content_language =
  content_type = application/octet-stream
  crc32c = NzUi7w==
  detect_md5hash = 3qWaTCGyUkCjOUnwQlTIJg==
  md5hash = 3qWaTCGyUkCjOUnwQlTIJg==
  name = butterfly01
  source = sample.eps
  storage_class = COLDLINE
google_storage_bucket_object.text:
  id = curry999904-butterfly02
  bucket = curry999904
  cache_control =
  content = dddddddddddddddddd
  content_disposition =
  content_encoding =
  content_language =
  content_type = text/plain; charset=utf-8
  crc32c = zTdyag==
  detect_md5hash = pBy73t+FQ949imvKMfWfVA==
  md5hash = pBy73t+FQ949imvKMfWfVA==
  name = butterfly02
  storage_class = COLDLINE

3.Delete Google Cloud Storage bucket

3.1.Correct the terraform tf file.

# cat gcs.tf
variable "cpath" { }
variable "project" { }
variable "region" { }

provider "google" {
  credentials  = "${file("${var.cpath}")}"
  project      = "${var.project}"
  region       = "${var.region}"
}

resource "google_storage_bucket" "COLDLINE" {
  name     = "curry999904"
  storage_class = "COLDLINE"
  location = "asia-northeast1"
}

resource "google_storage_bucket_object" "picture" {
  name   = "butterfly01"
  source = "sample.eps"
  bucket = "${google_storage_bucket.COLDLINE.name}"
}

resource "google_storage_bucket_object" "text" {
  name   = "butterfly02"
  content = "dddddddddddddddddd"
  bucket = "${google_storage_bucket.COLDLINE.name}"
}

3.2.Create terraform environment variable file.

# cat config.tfvars
cpath="/var/credentials/account.json"
project="projcet-crossbar-999999"
region="asia-northeast1"

3.3.Initialize Terraform.

# terraform init
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "google" (1.4.0)...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.google: version = "~> 1.4"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

3.4.Run Terraform.

# terraform destroy -force -var-file=config.tfvars
google_storage_bucket.COLDLINE: Refreshing state... (ID: curry999904)
google_storage_bucket_object.picture: Refreshing state... (ID: curry999904-butterfly01)
google_storage_bucket_object.text: Refreshing state... (ID: curry999904-butterfly02)
google_storage_bucket_object.picture: Destroying... (ID: curry999904-butterfly01)
google_storage_bucket_object.text: Destroying... (ID: curry999904-butterfly02)
google_storage_bucket_object.text: Destruction complete after 0s
google_storage_bucket_object.picture: Destruction complete after 0s
google_storage_bucket.COLDLINE: Destroying... (ID: curry999904)
google_storage_bucket.COLDLINE: Destruction complete after 1s

Destroy complete! Resources: 3 destroyed.

3.5.Check the status of terraform.

# terraform show

Summary

Refactoring is not executed.
Please refactor as necessary.
I would be pleased if it would be useful for those using Terraform.

“Upload objects to Google Cloud Storage using Terraform”. への1件のコメント

  1. hi, how could I create a directory in a GCP bucket using Terraform?

    thank you!

    いいね

コメントを残す