indexpost archiveatom feed syndication feed icon

Basic Auth with Python's xmlrpc.server


Just a quick note on how easy it turned out to be to add basic authentication to the Python standard library's xmlrpc.server

I previously played around with the XML RPC server in the standard library and wrote:

I think it wouldn't be too much trouble to extend the SimpleXMLRPCRequestHandler to support basic auth as a way to further limit access to the RPC server. Configuring the secrets on both the client and server sound like the exact case for LoadCredential which is built into systemd services.

I hadn't realized how easy that was going to be in practice, but I'm noting it here for future reference.

Basic authentication base64 encodes a username and password into an HTTP header and depends on it being present for every request. There is no particular implementation defined for how it is used. With this in mind the simplest possible thing seems to be to define a custom request handler to check the one HTTP method used by xmlrpc.server, a POST request.

from xmlrpc.server import SimpleXMLRPCRequestHandler
import base64

class AuthenticatingXMLRPCRequestHandler(SimpleXMLRPCRequestHandler):
    def do_POST(self):
        if not (cred := self.headers.get('Authorization')):
            self.wfile.write('no auth header received'.encode())
        elif cred == 'Basic ' + base64.b64encode('someuser:hunter2'.encode()).decode():

    def _reject_auth(self):
        self.send_header('WWW-Authenticate', 'Basic')
        self.send_header('Content-type', 'text/html')

That is technically sufficient. Requests from the client will be rejected without a matching username and password:

import xmlrpc.client

proxy = xmlrpc.client.ServerProxy('http://someuser:hunter2@localhost:9000')

Integrate with LoadCredential

Of course there is a lot of room to improve things with this example. Firstly it would be nice to not directly encode the username and password into the server code. How about integrating with LoadCredential? This might be preferable to something like an environment variable because environment variables can be inherited by subprocess or show up in CI output, debuggers, etc. Instead I'll encode a password in a way that it is safe to include directly into a service file, for convenience. systemd can automatically decrypt such values on service start and will expose the decrypted values as files which the service will read from.

This is a bit overwrought because a username and password is more likely to be drawn from a database or similar. It does demonstrate LoadCredential and that's what I happen to be interested in at the moment. First then is the service definition:

Description=an example

ExecStart=/usr/bin/python /opt/
SetCredentialEncrypted=example-password: \
        Whxqht+dQJax1aZeCGLxmiAAAAABAAAADAAAABAAAAC/9zj7A2sV/+oq/q4AAAAA6oguV \
        L4PkSdBaJ7ylFDAwgiZwBdCkYL5Ex7Ol09Ye55bxrox7CQSSXVZm8uHAynnfEWrSHBq7B \

This is sufficient to have systemd create a file at runtime named for the "key" of the credential, with the contents of the value and in the case of SetCredentialEncrypted the contents will be the decrypted value. The files are available at a directory whose name is available as the environment variable CREDENTIALS_DIRECTORY. The username portion is probably obvious, a a location $CREDENTIALS_DIRECTORY/some-user is a file containing the string "someuser". More interesting is the second encrypted example — where did this string come from?

# systemd-ask-password -n | systemd-creds encrypt --name=example-password -p - -

With these secrets available to the service it is possible to remove them from the server source and look them up during the the request handler class initialization. I picked this for convenience, recognizing once again that such credentials really make more sense on a per-request basis and checked in a database.

class AuthenticatingXMLRPCRequestHandler(xmlrpc.server.SimpleXMLRPCRequestHandler):
    cred_dir = os.getenv("CREDENTIALS_DIRECTORY")
    with open(f"{cred_dir}/example-user") as f:
        username =
    with open(f"{cred_dir}/example-password") as f:
        password =
    credential_string = "Basic " + base64.b64encode(f"{username}:{password}".encode()).decode()

    def do_POST(self):
        if not (cred := self.headers.get("Authorization")):
            self.wfile.write("no auth header received".encode())
        elif cred == self.credential_string:

    def _reject_auth(self):
        self.send_header("WWW-Authenticate", "Basic")
        self.send_header("Content-type", "text/html")

It is admittedly ugly. Rather than spend too long trying to clean up a single set of user credentials I feel satisfied that systemd credentials are pretty easy to use but not a great fit for this case.

Another thing to keep in mind though is that HTTP basic authentication is not worth much over plain HTTP. The base64 encoding doesn't really stand up to scrutiny and requires the addition of HTTPS to actually encrypt the connection and guard against eavesdropping.

Encrypt it with TLS

Once again I may be going overboard but let's add TLS to the connection between RPC client and server to secure the credential exchange. I'd like to keep most of what I already have in place including the socket activated nature of the service, dynamic users, etc.

I'll need a certificate and private key for this and I don't care about a full chain of trust so I'll make one myself.

$ openssl req -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -days 1
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [XX]:
State or Province Name (full name) []:
Locality Name (eg, city) [Default City]:
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:
Email Address []:

A few prompts out of the way and I've got a single certificate and key ready to go! I've just encrypted the key to include directly in the service file rather than creating a configuration directory for the encrypted file but this is just laziness:

# systemd-creds encrypt --name=certKey -p - key.pem

SetCredentialEncrypted=certCert: \
        c0a4KyC/Ese76tvFczP4j3KokEraO0XCb9a433xpzgFHlQhU5QWNLEEeN6fVRG/omO1lD \

The certificate can be encoded in a similar way but it isn't actually a secret.

With a certificate and key ready to encrypt things I need to extend the RPC server with TLS support. This turned out to be pretty easy with the ssl module from the standard library. The way to add TLS is to "wrap" a socket with a secure context - it turns out this works without issue using the socket passed in from systemd on socket activation. I need to provide the context with information about my certificate and key file location which means reading the CREDENTIALS_DIRECTORY environment variable again:

class ThreadedSimpleXMLRPCServer(socketserver.ThreadingMixIn,
    """A threaded, socket-activated SimpleXMLRPCServer"""
    def __init__(self, requestHandler):
                                                  (None, None),
        _socket = socket.fromfd(SYSTEMD_FIRST_SOCKET_FD, socket.AF_INET, socket.SOCK_STREAM)
        cred_dir = os.getenv('CREDENTIALS_DIRECTORY')
        context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH, cafile=certfile)
        context.check_hostname = False
        context.verify_mode = ssl.CERT_NONE
        context.load_cert_chain(certfile=certfile, keyfile=keyfile)
        self.socket = context.wrap_socket(_socket, server_side=True)

I've highlighted the only real changes necessary to enable TLS on the server (just 8 lines of code!). I was admittedly surprised at how straightforward this turned out to be. The certificate and key handling are as uninteresting as expected. Marginally more involved was finding the correct configuration for my server under a self-signed certificate. The ssl.Purpose.CLIENT_AUTH is important so that the server doesn't try mutually authenticating the connecting clients, which I won't be creating certificates for. I've disabled hostname checking because it was easier than recreating my certificate with a correct name but this seems tractable if I were doing this "for real". Similar to the client auth setting I've set the verify mode to not attempt verifying connecting clients.

On the client there is a similar process but even easier. The full client becomes:

import xmlrpc.client
import ssl

context = ssl.create_default_context(cafile='cert.pem')
context.check_hostname = False

proxy = xmlrpc.client.ServerProxy('https://someuser:hunter2@localhost:8081', context=context)

Where cert.pem is the public certificate used by the server, otherwise it will not verify for being an unknown certificate authority. Conveniently ServerProxy has support built in for secure contexts so it is only a matter of supplying it to the constructor. Having to pick a few of these values myself I was concerned about making some insecure selection and mistakenly disabling certificate checking. The create_default_context goes a long way towards simplifying this process. You may have noticed I created my certificate with an exceedingly short duration of just 1 day. I waited for it to expire and retested the above setup and receive the following when invoking the client:

$ python
Traceback (most recent call last):
  File "/home/nolan/sources/xmlrpc-example/", line 10, in <module>
  File "/usr/lib64/python3.11/xmlrpc/", line 1122, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python3.11/xmlrpc/", line 1464, in __request
    response = self.__transport.request(
  File "/usr/lib64/python3.11/xmlrpc/", line 1166, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python3.11/xmlrpc/", line 1178, in single_request
    http_conn = self.send_request(host, handler, request_body, verbose)
  File "/usr/lib64/python3.11/xmlrpc/", line 1291, in send_request
    self.send_content(connection, request_body)
  File "/usr/lib64/python3.11/xmlrpc/", line 1321, in send_content
  File "/usr/lib64/python3.11/http/", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.11/http/", line 1041, in _send_output
  File "/usr/lib64/python3.11/http/", line 979, in send
  File "/usr/lib64/python3.11/http/", line 1458, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/usr/lib64/python3.11/", line 517, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/lib64/python3.11/", line 1075, in _create
  File "/usr/lib64/python3.11/", line 1346, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1002)

Which is a satisfying thing to see! It seems my additions are encrypting the connection and I have not inadvertently disabled certificate checking entirely.


I probably need to raise my expectations because I continue to find myself surprised with how easy these things are. To go from nothing to a restricted, socket-activated, and TLS encrypted RPC server with HTTP basic authentication using just the standard library and the facilities provided by systemd in no time has been downright fun. I have previously considered doing TLS termination at the level of reverse proxies the "right way" to design things. TLS termination necessarily has implications for security behind the proxy and I'm now wondering if it isn't worth pursuing ideas like I've demonstrated here, with the encryption baked into the service itself. It imposes some overhead in both development and operation but some of these could be alleviated with better certificate management (like if each service had a certificate signed by a single, internal, trusted CA).

While basic authentication was pleasantly easy to layer over the server I think a more complete example supporting multiple users with a more dynamic credential store could be interesting. Multiple users or client certificates would motivate better logging. It might also be interesting to make this server more observable in general, the current logging is too opaque to be useful. Problems for another time perhaps!