Upload from a compute instance to OCI Object Storage: the CLI, an API key, and the instance principal that replaces it

Short answer: this is the lab where a compute instance stops being a place you log into and becomes a machine that talks to the cloud on its own. In about forty-five minutes you will install the OCI CLI on an instance, sign in the way the Console teaches you (an API key), upload a backup to a bucket, hit the permission error everyone hits, fix it, and then do the whole thing again the right way: as an instance principal, with no key on the machine at all. That last part is what separates a lab from production.

It follows on from our Object Storage lab, which created the bucket, shared a file and set lifecycle rules. You need that bucket, or any private bucket, before you start.

Before you start

• A compute instance you can SSH into. An Always Free VM.Standard.E2.1.Micro on Oracle Linux 9 is enough. The commands below are for Oracle Linux; the differences for Ubuntu are noted where they exist.
• A private bucket. The lab uses training-backups in a compartment called training; substitute yours.
• Your user must be able to create dynamic groups and policies in the tenancy, or have someone who can. In a classroom tenancy that is the trainer.
• Your tenancy's Object Storage namespace. You will find it in Step 3; write it down.

The concepts in one minute

• Who is calling. Every request to OCI is signed by an identity. When you run the CLI on your laptop, that identity is you, proven by an API key pair. When a machine runs the CLI unattended, it should be the machine, proven by an instance principal. Both work; only one is safe to leave running.
• What is allowed. Identity alone gets you nothing. A policy has to say that this user or this dynamic group may do this verb (read, use, manage) on this resource type (objects, buckets) in this compartment. No policy, no upload, and the error will not tell you why.
• Where the traffic goes. Object Storage is a public service, but an instance does not need a public IP to reach it. A service gateway on the VCN routes that traffic inside Oracle's network, so a backup host can sit in a private subnet with no internet access and still upload.
• Multipart. The CLI splits anything over 128 MiB into parts and uploads them in parallel, then joins them. You do not configure it; it just happens, which is why a 10 GB dump uploads the same way a 1 KB file does.

Step 1: make something worth backing up

Real backups are of real directories, so create one that looks like an application: a config file, a small data export and a log.

mkdir -p ~/app/config ~/app/data
echo "db_host=10.0.1.20" > ~/app/config/app.conf
seq 1 100000 > ~/app/data/orders.csv
dmesg 2>/dev/null | head -200 > ~/app/app.log || echo "startup ok" > ~/app/app.log

Now archive it with the date in the name, and record a checksum so you can prove the restore later.

STAMP=$(date +%F)
tar -czf ~/app-$STAMP.tar.gz -C ~ app
sha256sum ~/app-$STAMP.tar.gz | tee ~/app-$STAMP.sha256

Step 2: install the OCI CLI on the instance

Try it first; the instance may already have it.

oci --version

If the shell says the command is not found, install it from Oracle's own repository. On Oracle Linux 9:

sudo dnf -y install oraclelinux-developer-release-el9
sudo dnf -y install python39-oci-cli

On Oracle Linux 8 the packages are oraclelinux-developer-release-el8 and python36-oci-cli. On Ubuntu or any other distribution, Oracle's installer script does the same job in your home directory:

bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)"

Run oci --version again. A version number means the CLI is installed; it is not yet signed in as anyone.

Step 3: sign in with an API key, the way the Console teaches

This is the method every tutorial shows and the one most people leave running in production by accident. Do it once, so you know what it involves, and then replace it in Step 5.

oci setup config

The wizard asks for a config location (accept ~/.oci/config), your user OCID, your tenancy OCID and your region, then offers to generate a signing key pair. Accept. Both OCIDs are in the Console: click your profile icon at the top right, open My profile for the user OCID, and open Tenancy for the tenancy OCID.

The wizard leaves a public key at ~/.oci/oci_api_key_public.pem. Print it, and register it against your user in the Console: profile icon → My profile → API keys → Add API key → Paste a public key. The fingerprint the Console shows must match the one the wizard printed. Then tighten the file permissions the way the CLI wants them:

cat ~/.oci/oci_api_key_public.pem
oci setup repair-file-permissions --file ~/.oci/oci_api_key.pem
oci setup repair-file-permissions --file ~/.oci/config

Prove the sign-in works with the smallest possible call, which also gives you the namespace you will need for the rest of the lab:

oci os ns get

If that returns your namespace, the instance is now acting as you. Notice what just happened: a private key that can do everything your user can do is sitting in a file on a virtual machine.

Step 4: upload, and meet the error everyone meets

oci os object put --bucket-name training-backups \
  --file ~/app-$STAMP.tar.gz \
  --name backups/app/app-$STAMP.tar.gz

--bucket-name and --file are the only required parameters. --name sets the object name; without it the CLI uses the file name with no path. The backups/app/ prefix is not a folder, only part of the name, but the Console will display it as one.

In a classroom tenancy where the trainer has already written the policies, this returns the object's ETag and you are done. On a fresh tenancy, or a user with no rights on the bucket, you get a 404 whose code reads NotAuthorizedOrNotFound. OCI will not tell you which of the two it is; that is deliberate, so that an outsider cannot probe which buckets exist. Since the bucket exists, the cause is permission. The policy that fixes it, written by someone who can manage policies in the tenancy, is:

Allow group training-users to read buckets in compartment training
Allow group training-users to manage objects in compartment training where target.bucket.name='training-backups'

The first line lets the group see the bucket; the second lets it write objects, but only into that one bucket. Conditions like target.bucket.name are matched case-insensitively. Run the put again and check the result:

oci os object list --bucket-name training-backups --prefix backups/app/ \
  --query 'data[].{name:name, size:size, tier:"storage-tier"}' --output table

That table is your first upload. It worked as you, with your key on the machine. Now do it properly.

Step 5: the same upload as an instance principal

An instance principal lets the instance authenticate as itself. Oracle places a certificate on the instance and rotates it several times a day; there is nothing for you to create, store or rotate, and nothing to steal. Three pieces are needed, and they are all on the IAM side.

1. A dynamic group that matches the instance. In the Console: Identity & Security → Domains → your domain → Dynamic groups → Create dynamic group. Name it training-backup-hosts and give it a matching rule that selects every instance in the compartment:

Any {instance.compartment.id = 'ocid1.compartment.oc1..aaaaaaaa...'}

Paste the OCID of the training compartment. You can narrow the rule to a single instance with instance.id = '...', which is what you would do for one backup host in production.

2. A policy for that dynamic group, in the same compartment as the bucket:

Allow dynamic-group training-backup-hosts to read buckets in compartment training
Allow dynamic-group training-backup-hosts to manage objects in compartment training where target.bucket.name='training-backups'

Note that it is the same shape as the policy in Step 4, with dynamic-group in place of group. Dynamic groups get no rights by default, exactly like users.

3. The --auth flag on the instance. Upload the archive again under a new name, so you can see it arrive:

oci os object put --bucket-name training-backups \
  --file ~/app-$STAMP.tar.gz \
  --name backups/app/app-$STAMP-ip.tar.gz \
  --auth instance_principal

If you would rather not type the flag every time, set it once for the shell:

export OCI_CLI_AUTH=instance_principal
oci os object list --bucket-name training-backups --prefix backups/app/

A new dynamic group or policy can take a minute or two to apply. If the first attempt returns NotAuthorizedOrNotFound, wait and try once more before you start changing things.

Now remove the API key from the instance, because the whole point was to stop needing it:

rm ~/.oci/config ~/.oci/oci_api_key.pem ~/.oci/oci_api_key_public.pem
oci os ns get --auth instance_principal

The last command still returns the namespace. The machine has an identity of its own, and your user's key no longer exists on it. Delete that API key from your user in the Console as well; a key with no purpose is only a risk.

Step 6: make it run every night

A backup that depends on someone remembering it is not a backup. Save the upload as a script:

cat > ~/backup.sh <<'EOF'
#!/bin/bash
set -e
export OCI_CLI_AUTH=instance_principal
STAMP=$(date +%F)
tar -czf /tmp/app-$STAMP.tar.gz -C "$HOME" app
oci os object put --bucket-name training-backups \
  --file /tmp/app-$STAMP.tar.gz \
  --name backups/app/app-$STAMP.tar.gz \
  --storage-tier InfrequentAccess
rm -f /tmp/app-$STAMP.tar.gz
EOF
chmod +x ~/backup.sh

Two details matter. --storage-tier InfrequentAccess puts the backup straight into the cheaper tier, which is right for something read only when things go wrong. And the date in the name means each night's file is new; an existing name would make the CLI stop and ask before overwriting, which is not what you want at two in the morning (--force exists for the cases where you do mean to overwrite).

Schedule it with cron at 02:15 every night:

(crontab -l 2>/dev/null; echo "15 2 * * * $HOME/backup.sh >> $HOME/backup.log 2>&1") | crontab -

If the thing to back up is a whole directory tree rather than one archive, bulk-upload sends every file under it, keeping the relative paths as object names:

oci os object bulk-upload --bucket-name training-backups --src-dir ~/app --object-prefix raw/app/

Step 7: prove the restore

Nobody has a backup until they have restored from it. Download the archive to a new name and compare checksums:

oci os object get --bucket-name training-backups \
  --name backups/app/app-$STAMP.tar.gz \
  --file ~/restore-test.tar.gz
sha256sum ~/restore-test.tar.gz
cat ~/app-$STAMP.sha256
tar -tzf ~/restore-test.tar.gz | head

The two checksums match and the listing shows the files. That is the sentence you want to be able to say to an interviewer: not "we take backups" but "we restore-test them".

Step 8: clean up

Remove the cron line (crontab -e and delete it), and delete the test objects if the bucket is shared:

oci os object bulk-delete --bucket-name training-backups --prefix backups/app/ --force
oci os object bulk-delete --bucket-name training-backups --prefix raw/app/ --force

Keep the dynamic group and the policy: they are the reusable part, and in a real tenancy they are the pattern every backup host, log shipper and application server should use.

What you can now say in an interview

You installed the CLI on a compute instance, authenticated first as a user with an API key and then as an instance principal through a dynamic group and a scoped policy, explained why the second is the production pattern, read a NotAuthorizedOrNotFound error correctly, scheduled a nightly backup into the Infrequent Access tier, and restore-tested it with a checksum. Those are the Object Storage and IAM parts of the Architect Associate exam, and they are the questions a cloud administrator is actually asked in the first month on the job.

If you did this lab and want the rest of the platform taught the same way, with a mentor checking your work, the OCI program covers compute, networking, IAM, database and Object Storage over live sessions, and the Multi Cloud Administrator program does the same across OCI, AWS and Azure. Working professionals who want the automation side, Terraform and pipelines on OCI and Azure, will find it in the Cloud + DevOps program. Ask us for a call and a counsellor will tell you which fits.

Frequently asked questions