Added Terraform backend implementation for OCI Object Storage

[ravinitp]

Terraform backend implementation for OCI Object Storage

A new Terraform backend utilising Oracle Cloud Infrastructure (OCI) Object Storage as the storage backend. Leveraging OCI Object Storage provides a scalable and cost-effective alternative to traditional backends, offering a robust solution for state file storage and management.
RFC#34465
RFC#32634

Added

Terraform backend implementation for OCI Object Storage

Target Release

1.12.x

CHANGELOG entry

• This change is user-facing and I added a changelog entry.
• This change is not user-facing.

[ravinitp]

Test Case	Code	comand	Result	comment
only required attribute	terraform { backend "oci" { bucket = "state-file-bucket" namespace = "my-namespace" } }	./terraform apply	✅ Success
Missing required field	terraform { backend "oci" { namespace = "my-namespace" region = "us-ashburn-1" } }	./terraform init	Prompt	Enter the value of bucket
lock test with deferent machine with same state file in same bucket	-	./terraform apply on both machine	❌ One machine acquired the lock, another failed	Error: Error acquiring the state lock Lock Info: ID: 1fc42ad8-a338-906c-8781-c71b9eaf3c8f Path: tf-state-env/prod/etf.json Operation: OperationTypeApply Who: ravbhart@ravbhart-mac Version: 1.13.0 Created: 2025-04-10 15:35:23.842504 +0000 UTC Info:
lock test with deferent machine with different state file in the same bucket	terraform { backend "oci" { key= state1.json bucket = "state-file-bucket" namespace = "my-namespace" } } ------------------ terraform { backend "oci" { key= state2.json bucket = "state-file-bucket" namespace = "my-namespace" } }	./terraform apply on both machine at same time	✅ Both operation succeeded
optional attribute with basic auth	terraform { backend "oci" { bucket = "terraform-state" key = "etf.json" namespace = "namespace" tenancy_ocid = "ocid1.tenancy.oc1..xxxxx" user_ocid = "ocid1.user.oc1..xxxxx" fingerprint = "bd:9e:49:9c:39:1e:40:73:0c:91" private_key_path = "~/ssh/private_key_25-04-08T10_29_58.508Z.pem" region = "us-ashburn-1" } }	./terraform apply	✅ success
State Consistency Check Scenario Start a terraform apply operation from Machine A and keep it pending for approval. From Machine B, forcefully unlock the state file lock and initiate another terraform apply (this will overwrite the state). Go back to Machine A and approve the original pending operation.	-	-	❌ Unable to persist state file	ETag mismatch due to overwritten state
Corrupted state file scenario 1. Create a corrupted state file in the bucket. 2. Run terraform init and terraform apply. 3. Observe the error message.	-		❌ Unable to persist state file	│ Error: Unsupported state file format The state file could not be parsed as JSON: syntax error at byte offset 1.
new workspace env	-	./terraform workspace new dev ./terraform apply	✅ Success	After successful command able to see file tf-state-env/dev/terraform.tfstate and .md file
list workspace env	-	./terraform workspace list	✅ Success	Able to see list of workspace env default *dev
delete workspace	-	./terraform workspace select default ./terraform workspace delete dev	✅ Success	state file in dev env got deleted
Multipart Upload with Custom Part Size(not updatable for customers) Modified the DefaultFilePartSize to 1 KB (1024 bytes) to validate multipart upload functionality with small part sizes. This helps ensure that multipart logic triggers and functions correctly, even for small files.	-	./terraform apply	✅ Success	state file got uploaded using multipart approach Note: By default statefile > 128mb will use multipart upload.

[pvkrishnachaitanya]

Reviewed & Approved

[SarahFrench]

Hi @ravinitp, thanks for your PR!

Could you please let us know when this PR is ready for review, and mark the PR as draft until then? Force pushes to this branch make it hard for us to identify what code has changed since the last time we took a look.

[radeksimko]

@ravinitp Can you please run make syncdeps?

[ravinitp]

@radeksimko

@SarahFrench This is Instanceprincipal auth log
Uploading instancePrincipalAuth.log…

[ravinitp]

@radeksimko ,
I have addressed feedback from today's call

ravbhart-mac:oci ravbhart$ go test -v ./
=== RUN   TestBackendBasic
    backend_test.go:29: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-68010b5b"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
--- PASS: TestBackendBasic (16.90s)
=== RUN   TestBackendLocked_ForceUnlock
    backend_test.go:47: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-68010b6c"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
    backend_test.go:52: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-68010b6c"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
    testing.go:303: TestBackend: testing state locking for *oci.Backend
    backend_test.go:61: TestBackend: testing state locking for *oci.Backend
    testing.go:303: TestBackend: testing state locking for *oci.Backend
    backend_test.go:63: TestBackend: testing state locking for *oci.Backend
--- PASS: TestBackendLocked_ForceUnlock (30.42s)
=== RUN   TestBackendBasic_multipart_Upload
    backend_test.go:75: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-68010b8a"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
--- PASS: TestBackendBasic_multipart_Upload (19.35s)
=== RUN   TestOCIBackendConfig_PrepareConfigValidation
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/null_bucket
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/key_with_trailing_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/key_with_double_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/workspace_key_prefix_with_leading_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/encryption_key_conflict
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/invalid_auth_method
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/empty_bucket
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/null_namespace
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/empty_namespace
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/key_with_leading_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/Invalid_encryption_key_combination
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/private_key_and_private_key_path_conflict
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/missing_region_for_InstancePrinciple_auth
--- PASS: TestOCIBackendConfig_PrepareConfigValidation (0.01s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/null_bucket (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/key_with_trailing_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/key_with_double_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/workspace_key_prefix_with_leading_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/encryption_key_conflict (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/invalid_auth_method (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/empty_bucket (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/null_namespace (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/empty_namespace (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/key_with_leading_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/Invalid_encryption_key_combination (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/private_key_and_private_key_path_conflict (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/missing_region_for_InstancePrinciple_auth (0.00s)
PASS
ok  	github.com/hashicorp/terraform/internal/backend/remote-state/oci	70.510s
ravbhart-mac:oci ravbhart$ 

[radeksimko]

@ravinitp I originally thought the dependency change in the consul backend was unrelated to this PR but it is. I assume the objx library is a shared transitive dependency and as a result of OCI being added, it impacts consul's go.sum.

With that in mind, can you please run make syncdeps and commit and push that change to the consul backend?

[radeksimko]

@ravinitp As per CI failure - Can you also run the following command and commit & push the outcome, please?

go tool golang.org/x/tools/cmd/goimports -w ./internal/backend/remote-state/oci

[ravinitp]

Done goimports and sync deps

ravbhart-mac:oci ravbhart$ go test -v ./
=== RUN   TestBackendBasic
    backend_test.go:30: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-68012121"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
--- PASS: TestBackendBasic (15.57s)
=== RUN   TestBackendLocked_ForceUnlock
    backend_test.go:48: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-68012131"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
    backend_test.go:53: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-68012131"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
    testing.go:303: TestBackend: testing state locking for *oci.Backend
    backend_test.go:62: TestBackend: testing state locking for *oci.Backend
    testing.go:303: TestBackend: testing state locking for *oci.Backend
    backend_test.go:64: TestBackend: testing state locking for *oci.Backend
--- PASS: TestBackendLocked_ForceUnlock (29.58s)
=== RUN   TestBackendBasic_multipart_Upload
    backend_test.go:76: TestBackendConfig on *oci.Backend with configtesting.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-oci-test-6801214e"), "key":cty.StringVal("testState.json"), "namespace":cty.StringVal("idxvx0yktto5")}}
--- PASS: TestBackendBasic_multipart_Upload (19.15s)
=== RUN   TestOCIBackendConfig_PrepareConfigValidation
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/null_bucket
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/null_namespace
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/key_with_double_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/workspace_key_prefix_with_leading_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/encryption_key_conflict
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/Invalid_encryption_key_combination
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/private_key_and_private_key_path_conflict
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/invalid_auth_method
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/empty_bucket
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/empty_namespace
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/key_with_leading_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/key_with_trailing_slash
=== RUN   TestOCIBackendConfig_PrepareConfigValidation/missing_region_for_InstancePrinciple_auth
--- PASS: TestOCIBackendConfig_PrepareConfigValidation (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/null_bucket (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/null_namespace (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/key_with_double_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/workspace_key_prefix_with_leading_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/encryption_key_conflict (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/Invalid_encryption_key_combination (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/private_key_and_private_key_path_conflict (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/invalid_auth_method (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/empty_bucket (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/empty_namespace (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/key_with_leading_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/key_with_trailing_slash (0.00s)
    --- PASS: TestOCIBackendConfig_PrepareConfigValidation/missing_region_for_InstancePrinciple_auth (0.00s)
PASS
ok  	github.com/hashicorp/terraform/internal/backend/remote-state/oci	65.652s
ravbhart-mac:oci ravbhart$ 

[crw]

IMO, the term oci should be reserved for future support of the Open Container Initiative, since that’s what OCI commonly refers to for most people outside the Oracle Cloud ecosystem.

Thanks for raising this! Given that the provider is already named oci (https://registry.terraform.io/providers/oracle/oci/latest/docs), we will continue to use the un-prefixed nomenclature for this backend. Thanks again for bringing this to our attention.