As some of you may remember, I've been working on an effort to include SMB, via
Samba, in the container ecosystem and Kubernetes. It's been a while since I
wrote anything about these projects. I want to give an update on our work to
keep others informed, and possibly get others interested in what we've been
doing. I hope to make this a regular thing (unless you get annoyed and tell
me to stop ;-) ).
We gave a presentation on these projects at sambaXP and also have had a few
posts on the samba mailing list about them. Since then we've been doing a lot
of varied work and below is a small slice of it.
CTDB
I took on the task of trying to "containerize" CTDB-enabled Samba over this
summer. With assistance of the samba-technical mailing list we successfully
started running CTDB in a containerized environment. I proceeded to automate
parts of the CTDB configuration in our sambacc library . Finally, we
created new example YAML files in samba-containers that demonstrate a
clustered instance of Samba running under Kubernetes orchestration.
There were a lot of challenges getting CTDB working automatically in a
container and there's still a lot to do. For example, the code I wrote to help
manage the CTDB nodes file isn't as robust as I'd like it to be and is still
generally immature. We'd like to discuss ways we might update CTDB to make the
nodes easier to manage through automation (in our environment) and hope such
discussions could improve things for other use cases as well.
At the moment, we've been using a Kubernetes based mechanism for getting
clients running outside of the cluster access to the shares. Without plunging
deep into the particulars of Kubernetes networking; the method works well
enough but does not use the CTDB public address mechanism. As such, we feel the
failover behavior we have right now may not be as nice at getting clients to
quickly fail-over when compared to CTDB's tickle-ack. This is an area we're
going to be actively investigating and would like to hear additional feedback
on this topic.
We're also adding support for the CTDB enabled Samba instances to the
Kubernetes operator code. The plan is to continue to use the single SMB
instance by default but add an experimental mode for running a StatefulSet of
Pods that include CTDB for clustering across multiple nodes.
We are also discussing the need for using the fileid vfs module on top the
clustered file system we intend to use (cephfs). However, we don't want to
require a PVC to be any particular file system and so we have been discussing
ways to enable the fileid module, and how to configure it, only in cases that
we need it in an easy-to-use manner.
ACL Xattr
We are very interested in being able to use NTACLs on the shares that we host
in containerized Samba servers. Today, the acl_xattr module stores the NTACL
metadata in an xattr named "security.NTACL". The "security" namespaces on Linux
requires the use of the CAP_SYS_ADMIN capability - basically the "root user" of
Linux capabilities. This would require running the containers with escalated
privileges; something we would prefer not to do. So, Günther Deschner, with
feedback from Ralph Böhme and Michael Adam and others, has been working on the
patches in MR#1908 .
With these changes in place, we'd be able to configure Samba with a custom
xattr for our containerized Samba instances and thus avoid needing to run a
privileged container while still supporting NTACLs.
Nightly Builds
We've been looking in an easy way to try out the latest code in Samba and are
planning on adding support for building container images based on nightly
package builds. Anoop C S has already added support to the sambacc layer in
order to support the newer CLI options present in Samba 4.15. Next we plan to
add new build rules to the samba-containers project that will create new build
tags in our container registry repo(s). Soon we should be able to more easily
test and experiment with the latest versions Samba has to offer!
Metrics
In the slightly longer term, we would like to add metrics support to our
pods running in Kubernetes. Our plan is to follow the de-facto standard
in this space and export Prometheus metrics. In order to do this we need
to get the data out of our Samba stack and rather than try and directly
scrape the output of a tool like smbstatus we are very interested in the
effort to add JSON output support to samba tools. In our case, we want JSON
output from smbstatus. As such we're very interested in MR#1269 .
We'll probably be getting more involved here in the near future.
Offline Domain Join
One new feature that landed in Samba 4.15 is the new Offline Domain Join (ODJ)
support. Currently, to connect our containers with Active Directory we're
asking users to store a username/password in a Kubernetes secret . We are
aware that storing a password (even within a secret) isn't the best thing
to do, so we're looking forward to trying to integrate the ODJ feature
into our projects so that users never have to store a password.
Watch this space! :-)
Wrap Up
If you have any questions or comments feel free to ask! I'm hoping this
update both serves to inform you of what we're up to as well as serving as
a prompt for you to give us feedback.
Thanks for reading!
