All About File Systems, Freeware and Features

Featured article by Stefan Bernbo, founder and CEO, Compuverde

More is being asked of data storage than at any time before. Streaming video, cloud-based business applications and Big Data are among the forces requiring a rethink of traditional storage architectures. Scaling linearly with legacy storage appliances is far too expensive – and it would take too long. Even adding multiple servers could not accommodate storage demands. Vertical storage architecture contains bottlenecks that slow performance to an unacceptable level.

Help has arrived to overcome these challenges in the form of software-defined storage (SDS). Since SDS decouples the programming that controls storage-related tasks from the physical storage hardware, it dramatically reduces costs associated with hardware. Fewer, less-expensive servers can be used to improve both capacity and performance. Administration is simplified and made more flexible and efficient. SDS enables users to allocate and share storage assets across all workloads.

SDS has become widely adopted, consequently. Gartner recently reported that by 2020, anywhere from 70 to 80 percent of unstructured data will be stored and managed on lower-cost hardware supported by software-defined storage.

Why You Need File Systems

Because 80 percent of data is unstructured, storage solutions that offer file systems currently represent 80 percent of the market. While it is widely understood that unstructured data is best managed with a file system, for some reason, many SDS offerings focus solely on block or object store. Few focus on file systems or do them well. Without a file system overlaying this data, it becomes very difficult to manage that data.

There are three categories of storage, each with its particular function:

– Block is a standard go-to for storage. It is used for storing databases or virtual machines, but you need files as well to deal with all the unstructured data.

– Object storage is getting all the attention lately. It is used for machine-to-machine/IoT transactions and other applications that require extreme scalability, but it isn’t that much better than block when it comes to managing data.

– File systems are not as exciting as object, perhaps, but they are the best at handling unstructured data.

File systems are critical; everyone understands that, at least to a degree. Some software-defined storage providers claim to provide a file system with their offerings. However, these file systems are usually based on Samba and exclude some features most Windows users are used to.

Of those who have realized that their vendor’s file system solution doesn’t cut muster,

many have turned to Samba, the freeware module that enables support for SMB and allows end users to access and use files on the company’s intranet or network. However, providing file services through Samba, which is open source, often means going without needed features.

This isn’t about a “nice to have” list; the importance of features can’t be underestimated. It’s not just the file system that organizations need in order to deal with unstructured data; file-related features are also necessary. These include:

– Quota: This feature helps you monitor the amount of storage being used. You can set a soft limit quota that will warn you when part of a file system is close to reaching its storage limit, but still allow data to be saved. If you set up a hard limit quota, after the quota is reached, no new data can be saved.

– Retention: You can use this feature to automatically create a single folder or a hierarchy of folders on file servers, to be deleted according to assigned policies.

– Tiering: With tiering, a policy enables you to designate where a specific file will be placed and if and when the file will be migrated between file system pools. You can define both file placement and migration policies. By using a policy, you create a filter which designates a specific file type to a particular tier. Tiered storage is more efficient and boosts performance.

– Snapshot: This feature is a read-only copy of the contents of a file system or independent file set taken at a single point in time. When a snapshot of an independent file set is taken, all files and nested dependent file sets will be included in the snapshot.

A Complete Storage Solution

It is a sad irony to be in need of a file system to manage unstructured data yet notice that the majority of SDS providers have very little to offer in this respect. Though some take the open source approach, they soon discover that its feature sets are limited. A robust storage solution is one that takes a comprehensive approach that welcomes all storage types: object, block and file system. Research your options carefully to ensure that you don’t end up with a feature-poor, token file system that will send you back to storage square one.

About the Author

Stefan Bernbo is the founder and CEO of Compuverde. For 20 years, Stefan has designed and built numerous enterprise-scale data storage solutions designed to be cost effective for storing huge data sets. From 2004 to 2010 Stefan worked within this field for Storegate, the wide-reaching Internet based storage solution for consumer and business markets, with the highest possible availability and scalability requirements. Previously, Stefan has worked with system and software architecture on several projects with Swedish giant Ericsson, the world-leading provider of telecommunications equipment and services to mobile and fixed network operators.

About the Author

Stefan Bernbo is the founder and CEO of Compuverde. For 20 years, Stefan has designed and built numerous enterprise scale data storage solutions designed to be cost effective for storing huge data sets. From 2004 to 2010 Stefan worked within this field for Storegate, the wide-reaching Internet based storage solution for consumer and business markets, with the highest possible availability and scalability requirements. Previously, Stefan has worked with system and software architecture on several projects with Swedish giant Ericsson, the world-leading provider of telecommunications equipment and services to mobile and fixed network operators.