My recent work on integrating Microsoft Azure File Storage with Kubernetes storage is available for testing.
Azure File Storage is basically a SMB 3.0 file share. Each time a VM needs a file share, you can use your storage account to create one. There is one limitation for Linux VM: for lack of encryption in kernel CIFS implementation, Linux VM must colocate with the file share in the same Azure region. Thus, for now Kubernetes hosts must live in Azure Computing VMs to access their Azure file share.
It is also possible to use Azure Block Blob storage for Kubernetes, though that’ll require more efforts and new APIs from Azure.
Throughout my career, I have dealt with permission denied problems. While root cause could vary, they more or less fall into the followings.
The Unix ACL defines read/write/execute privileges for owner, group member, and others. One can use chmod(1) to protect files and directories. Users that don’t have the proper privileges are denied of access.
Linux Capabilities defines capabilities that executables have for certain actions. For example, a program needs CAP_CHOWN to execute chown(2). Similarly, a container that wants to run mount has to have CAP_SYS_ADMIN capability to avoid permission deny problem, which can be supplied by –cap-add option to docker run command.
SELinux is a role based mandatory access control. SELinux is supported by many filesystems. Once SELinux is enabled, a filesystem must be mounted with proper label to allow processes to access content. For example, a Docker container needs svirt_sandbox_file_t label to access external volume.
Some Special Cases
- root_squash, and all_squash. An NFS server exports a share with root_squash strips uid 0 and converts it to anonymous id. All_squash converts any uid to anonymous id.
- NFS v4 ACL. v4 ACL defined rules for read/write/execute/chown/delete/etc. Missing flags also cause permission deny.
Setting up a Ceph cluster could be painful for some. Fortunately, there are good tutorials and tools around.
Ceph storage cluster installation and HowTo CEPH Quick Start/Installation Ubuntu VIDEO both walk overall process.
Ceph-ansible automates the setup process. It creates a all-in-one Ceph cluster using Vagrant and runs on VirtualBox and (expected to run on) Docker and bare mental.