This article is a general introduction to Swift, OpenStack’s Object Storage.\u00a0 We will cover the following topics: what Swift is, what kind of data you can store in Swift, how Swift controls access to your data, how Swift protects that data using replication or erasure codes, where to find development resources for Swift? and much more.<\/p>\n
Conceptually, Swift is similar to Amazon’s S3 or Microsoft Azure’s Blob Storage.\u00a0 It allows you to store any kind of unstructured data using a web-based API.<\/p>\n
Swift is designed to scale to billions of objects and petabytes of storage.\u00a0 Swift does not require any special hardware.\u00a0 It runs on commodity servers with normal disks.\u00a0 However, plugins allow it to use filers from NetApp, Solidfire, and others if so desired.<\/p>\n
The Swift software uses a distributed, shared nothing model.\u00a0 Since there is no central database or node to act as a bottleneck, it has tremendous horizontal scaling.\u00a0 To get more capacity, you simply have to add more nodes.\u00a0 Swift stores the data redundantly. \u00a0 If a server or disk fails, Swift uses the data on the surviving hardware to restore the data to its full level of redundancy.<\/p>\n
Physically, Swift is set-up as a set of nodes in a cluster.\u00a0 A proxy node receives incoming requests and directs them to storage nodes where the actual data resides.\u00a0 OpenStack recommends a minimum of 5 storage nodes<\/a> for production use, so Swift takes some hardware to set-up.<\/p>\n Swift stores its objects in containers.\u00a0 Each user’s storage account can have as many containers as they want.\u00a0 Each object is associated with a single container.\u00a0 An object is referenced by a URL of the form:<\/p>\n https:\/\/<storage_account_name>\/<container name>\/<object><\/p>\n One thing to note is that containers cannot be nested.\u00a0 You cannot have a container within a container.\u00a0 But you can simulate a pseudo-hierarchy<\/a>, albeit with some extra effort, by adding slashes in the objects name and doing special list<\/em> operations.<\/p>\n Pretty much anything you want.\u00a0 An object can contain a document, photos, music, back-ups, VM images, or any other piece of unstructured data.\u00a0 Swift is set-up to let you access individual objects individually via their URLs.\u00a0 It does not organize the data into tables like HBase or store relationships between data like MySQL.\u00a0 Nor is it a filesystem.\u00a0 A container name is the only organization that it offers to the data.<\/p>\n That said, Swift is very powerful.\u00a0 Since the data is accessible via a URL, it is easy to access it from anywhere on the Internet.\u00a0 It can be easily accessed from cell phones, PCs, or a variety of devices.\u00a0 And the ability to store objects can mesh very nicely with object oriented programming when it needs to persist data.<\/p>\n Although Swift can be run stand-alone, most commonly it is run as part of OpenStack.\u00a0 In this configuration, authentication is done through Keystone, the OpenStack Authentication Server.\u00a0 Once the user has presented credentials to Keystone, he is given an authentication token which is valid for 24 hours.\u00a0 This token proves that the user is the owner of the Swift storage account.\u00a0 By default, only the owner can access the account.<\/p>\n The owner can define individual Access Control Lists (ACLs) for each container.\u00a0 The owner can grant read and\/or write access to everyone, various groups, referrer hosts, or domains.\u00a0\u00a0 For more details, see the ACL section<\/a> of the API doc.<\/p>\n Finally, Swift has a temporary URL feature.\u00a0 This allows a third party to access an object using a URL that will expire in within a certain amount of time.\u00a0 This is useful for doing things like mailing temporary download links to people.<\/p>\n By default, Swift replicates each object three times.\u00a0 When storing the copies, it tries to spread them out over different servers and disks so the failure of a single component won’t cause lost data.\u00a0 The number of replicas for each object is configurable by the administrator.<\/p>\n Erasure codes use less storage than replicas.\u00a0 Rather than duplicating the complete object multiple times, parity data is created much like RAID.\u00a0 This can reduce the amount of storage for an object from 3X to 1.2X.\u00a0 For cold data which is simply being archived, this can be a tremendous win.\u00a0 Rackspace has a more detailed description of Swift erasure codes here<\/a>.<\/p>\n Erasure coding isn’t in Swift yet, but the development community will be working on it throughout this year.<\/p>\n The OpenStack foundation has online documentation for Swift<\/a>.\u00a0 The Associated Projects<\/a> page for Swift has API bindings for various languages.\u00a0 The only officially supported binding is for Python, but there are others for PHP, Java, and C#.\u00a0 Finally, if you have questions, you can always ask the OpenStack community<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"\u00a0What Can You Store in Swift?<\/span><\/strong><\/h3>\n
\u00a0How Swift Controls Access to Your Data<\/strong><\/span><\/h3>\n
\u00a0How Swift Protects Data Using Replication or Erasure Codes<\/strong><\/span><\/h3>\n
Where to Find Development Resources for Swift<\/span><\/strong><\/h3>\n