Object Storage

OpenStack Object Storage (Swift) service offers “object storage as a service”. Swift is used for redundant, scalable data storage using clusters of standardized servers to store petabytes of accessible data. It uses a distributed architecture with no central point of control, providing greater scalability, redundancy, and permanence, and it ensures data replication and integrity across the cluster.

It is a long-term storage system for large amounts of static data which can be retrieved and updated, and it is ideal for storing unstructured data that can grow without bound.

Swift also provides a fully distributed, API-accessible storage platform that can be integrated directly into applications or used for backup, archiving, and data retention.

Swift Concepts

The Object Storage system organizes data in a hierarchy, as follows:

• Account: Represents the top-level of the hierarchy. In the OpenStack environment, account is synonymous with a project or tenant. The account defines a namespace for containers. A container might have the same name in two different accounts.

• Container: Defines a namespace for objects. An object with the same name in two different containers represents two different objects. You can create any number of containers within an account. In addition to containing objects, you can also use the container to control access to objects by using an access control list (ACL). You cannot store an ACL with individual objects. In addition, you configure and control many other features, such as object versioning, at the container level. You can set a storage policy on a container with predefined names and definitions from your cloud provider.

• Object: Stores data content, such as documents, images, and so on. You can also store custom metadata with an object. With the Object Storage API, you can:

  • Store an unlimited number of objects. Each object can be as large as 5 GB, which is the default. You can configure the maximum object size.
  • Upload and store objects of any size with large object creation.
  • Use cross-origin resource sharing to manage object security.
  • Compress files using content-encoding metadata.
  • Override browser behavior for an object using content-disposition metadata.
  • Schedule objects for deletion.
  • Bulk-delete up to 10,000 objects in a single request.
  • Auto-extract archive files.
  • Generate a URL that provides time-limited GET access to an object.
  • Upload objects directly to the Object Storage system from a browser by using form POST middleware.
  • Create symbolic links to other objects.

The account, container, and object hierarchy affects the way you interact with the Object Storage API. Specifically, the resource path reflects this structure and has this format:

/v1/{account}/{container}/{object}

For example, for the flowers/rose.jpg object in the images container in the 12345678912345 account, the resource path is:

/v1/12345678912345/images/flowers/rose.jpg

Notice that the object name contains the / character. This slash does not indicate that Object Storage has a sub-hierarchy called flowers because containers do not store objects in actual sub-folders. However, the inclusion of / or a similar convention inside object names enables you to create pseudo-hierarchical folders and directories

You will find more information about the Swift API here: https://docs.openstack.org/swift/latest/api/object_api_v1_overview.html

Swift and AWS S3 comparison

General compatibility statement. S3 is a product from Amazon, and as such, it includes “features” that are outside the scope of Swift itself. For example, even it is possbile provide that functionality, log delivery is a service outside of Swift. There are also other features not supported by the Swift’s API like: Versioning, Bucket Policy, Lifecycle and Accelerate (CDN), or serverless Public Website.

Object Storage operations via Horizon

Containter creation

Login with your Tecnalia account via Web Access -- Horizon OpenStack Service. (https://cloud.tecnalia.dev)

On panel Project -> Object Storage -> Containers Click on "+ Container".

Fill out the name field and Select the Container Access "Public or no Public".

info

A public container will allow anyone with the Public URL to gain access to your objects in the container

Operations into a Container

Login with your TRI account via Web Access -> Horizon OpenStack Service.

On panel Project -> Object Storage -> Containers

Create a folder into a container (+Folder)

Upload a file into a container

Download a file from a container

Object Storage operations via CLI

Container creation

Use your app credentials. For more information, goes to subsection 3.2 CLI Access:

source app-cred-openrc.sh

Get a token. It will be necessary for any swift operations

openstack token issue

Field	Value
expires	2021-05-25T10:15:18+0000
id	token-id
project_id	project-id
user_id	user-id

Create a container

openstack container create container-01

List your account

openstack container show container-01

Field	Value
account	AUTH_account
bytes_used	8063
container	container-01
object_count	1

List containers

openstack container list

Name
container-01

Uploads files or directories

openstack object create container-01 folder-01/file-01.txt

object	container	etag
folder-01/file-01.txt	container-01	d41d8cd98f00b204e9800998ecf8427e

List container objects

openstack object list container-01

Name
folder-01/file-01.txt

Download a file

openstack object save container-01 folder-01/file-01.txt