Just a quick note on how easy it turned out to be to add basic
authentication to the Python standard
library's xmlrpc.server
I previously played around with the XML RPC server in the standard library and wrote:
I think it wouldn't be too much trouble to extend theSimpleXMLRPCRequestHandlerto support basic auth as a way to further limit access to the RPC server. Configuring the secrets on both the client and server sound like the exact case forLoadCredentialwhich is built into systemd services.
I hadn't realized how easy that was going to be in practice, but I'm noting it here for future reference.
Basic authentication base64 encodes a username and password into an
HTTP header and depends on it being present for every request. There
is no particular implementation defined for how it is used. With
this in mind the simplest possible thing seems to be to define a
custom request handler to check the one HTTP method used by
xmlrpc.server,
a POST request.
from xmlrpc.server import SimpleXMLRPCRequestHandler
import base64
class AuthenticatingXMLRPCRequestHandler(SimpleXMLRPCRequestHandler):
def do_POST(self):
if not (cred := self.headers.get('Authorization')):
self._reject_auth()
self.wfile.write('no auth header received'.encode())
elif cred == 'Basic ' + base64.b64encode('someuser:hunter2'.encode()).decode():
super().do_POST()
else:
self._reject_auth()
def _reject_auth(self):
self.send_response(401)
self.send_header('WWW-Authenticate', 'Basic')
self.send_header('Content-type', 'text/html')
self.end_headers()That is technically sufficient. Requests from the client will be rejected without a matching username and password:
import xmlrpc.client
proxy = xmlrpc.client.ServerProxy('http://someuser:hunter2@localhost:9000')LoadCredential
Of course there is a lot of room to improve things with this
example. Firstly it would be nice to not directly encode the
username and password into the server code. How about integrating
with LoadCredential? This might be preferable to
something like an environment variable because environment variables
can be inherited by subprocess or show up in CI output, debuggers,
etc. Instead I'll encode a password in a way that it is safe to
include directly into a service file, for convenience. systemd can
automatically decrypt such values on service start and will expose
the decrypted values as files which the service will read from.
This is a bit overwrought because a username and password is more
likely to be drawn from a database or similar. It does
demonstrate LoadCredential and that's what I happen to
be interested in at the moment. First then is the service
definition:
[Unit]
Description=an example
Requires=example.socket
[Service]
ExecStart=/usr/bin/python /opt/auth-server.py
SetCredential=example-user:someuser
SetCredentialEncrypted=example-password: \
Whxqht+dQJax1aZeCGLxmiAAAAABAAAADAAAABAAAAC/9zj7A2sV/+oq/q4AAAAA6oguV \
L4PkSdBaJ7ylFDAwgiZwBdCkYL5Ex7Ol09Ye55bxrox7CQSSXVZm8uHAynnfEWrSHBq7B \
5dMCr0l9jE
This is sufficient to have systemd create a file at runtime named
for the "key" of the credential, with the contents of the value and
in the case of SetCredentialEncrypted the contents will
be the decrypted value. The files are available at a directory whose
name is available as the environment
variable CREDENTIALS_DIRECTORY. The username portion is
probably obvious, a a
location $CREDENTIALS_DIRECTORY/some-user is a file
containing the string "someuser". More interesting is the second
encrypted example — where did this string come from?
# systemd-ask-password -n | systemd-creds encrypt --name=example-password -p - -With these secrets available to the service it is possible to remove them from the server source and look them up during the the request handler class initialization. I picked this for convenience, recognizing once again that such credentials really make more sense on a per-request basis and checked in a database.
class AuthenticatingXMLRPCRequestHandler(xmlrpc.server.SimpleXMLRPCRequestHandler):
cred_dir = os.getenv("CREDENTIALS_DIRECTORY")
with open(f"{cred_dir}/example-user") as f:
username = f.read()
with open(f"{cred_dir}/example-password") as f:
password = f.read()
credential_string = "Basic " + base64.b64encode(f"{username}:{password}".encode()).decode()
def do_POST(self):
if not (cred := self.headers.get("Authorization")):
self._reject_auth()
self.wfile.write("no auth header received".encode())
elif cred == self.credential_string:
super().do_POST()
else:
self._reject_auth()
def _reject_auth(self):
self.send_response(401)
self.send_header("WWW-Authenticate", "Basic")
self.send_header("Content-type", "text/html")
self.end_headers()It is admittedly ugly. Rather than spend too long trying to clean up a single set of user credentials I feel satisfied that systemd credentials are pretty easy to use but not a great fit for this case.
Another thing to keep in mind though is that HTTP basic authentication is not worth much over plain HTTP. The base64 encoding doesn't really stand up to scrutiny and requires the addition of HTTPS to actually encrypt the connection and guard against eavesdropping.
Once again I may be going overboard but let's add TLS to the connection between RPC client and server to secure the credential exchange. I'd like to keep most of what I already have in place including the socket activated nature of the service, dynamic users, etc.
I'll need a certificate and private key for this and I don't care about a full chain of trust so I'll make one myself.
$ openssl req -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -days 1
...
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:
State or Province Name (full name) []:
Locality Name (eg, city) [Default City]:
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:
Email Address []:
A few prompts out of the way and I've got a single certificate and key ready to go! I've just encrypted the key to include directly in the service file rather than creating a configuration directory for the encrypted file but this is just laziness:
# systemd-creds encrypt --name=certKey -p - key.pem
SetCredentialEncrypted=certCert: \
Whxqht+dQJax1aZeCGLxmiAAAAABAAAADAAAABAAAAC/qMMgwwjiijXqAEIAAAAAl8WsQ \
c0a4KyC/Ese76tvFczP4j3KokEraO0XCb9a433xpzgFHlQhU5QWNLEEeN6fVRG/omO1lD \
...
The certificate can be encoded in a similar way but it isn't actually a secret.
With a certificate and key ready to encrypt things I need to extend
the RPC server with TLS support. This turned out to be pretty easy
with
the ssl
module from the standard library. The way to add TLS is to
"wrap" a socket with a secure context - it turns out this works
without issue using the socket passed in from systemd on socket
activation. I need to provide the context with information about my
certificate and key file location which means reading
the CREDENTIALS_DIRECTORY environment variable again:
class ThreadedSimpleXMLRPCServer(socketserver.ThreadingMixIn,
xmlrpc.server.SimpleXMLRPCServer):
"""A threaded, socket-activated SimpleXMLRPCServer"""
def __init__(self, requestHandler):
xmlrpc.server.SimpleXMLRPCServer.__init__(self,
(None, None),
requestHandler=requestHandler,
bind_and_activate=False)
SYSTEMD_FIRST_SOCKET_FD = 3
_socket = socket.fromfd(SYSTEMD_FIRST_SOCKET_FD, socket.AF_INET, socket.SOCK_STREAM)
cred_dir = os.getenv('CREDENTIALS_DIRECTORY')
keyfile=f"{cred_dir}/certKey"
certfile=f"{cred_dir}/certCert"
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH, cafile=certfile)
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE
context.load_cert_chain(certfile=certfile, keyfile=keyfile)
self.socket = context.wrap_socket(_socket, server_side=True)
I've highlighted the only real changes necessary to enable TLS on
the server (just 8 lines of code!). I was admittedly surprised at
how straightforward this turned out to be. The certificate and key
handling are as uninteresting as expected. Marginally more involved
was finding the correct configuration for my server under a
self-signed certificate. The ssl.Purpose.CLIENT_AUTH is
important so that the server doesn't try mutually authenticating the
connecting clients, which I won't be creating certificates for. I've
disabled hostname checking because it was easier than recreating my
certificate with a correct name but this seems tractable if I were
doing this "for real". Similar to the client auth setting I've set
the verify mode to not attempt verifying connecting clients.
On the client there is a similar process but even easier. The full client becomes:
import xmlrpc.client
import ssl
context = ssl.create_default_context(cafile='cert.pem')
context.check_hostname = False
proxy = xmlrpc.client.ServerProxy('https://someuser:hunter2@localhost:8081', context=context)
Where cert.pem is the public certificate used by the
server, otherwise it will not verify for being an unknown
certificate authority. Conveniently ServerProxy has
support
built in for secure contexts so it is only a matter of supplying
it to the constructor. Having to pick a few of these values myself I
was concerned about making some insecure selection and mistakenly
disabling certificate
checking. The create_default_context goes a long way
towards simplifying this process. You may have noticed I created my
certificate with an exceedingly short duration of just 1 day. I
waited for it to expire and retested the above setup and receive the
following when invoking the client:
$ python auth-client.py
Traceback (most recent call last):
File "/home/nolan/sources/xmlrpc-example/auth-client.py", line 10, in <module>
print(proxy.some_call())
^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/xmlrpc/client.py", line 1122, in __call__
return self.__send(self.__name, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/xmlrpc/client.py", line 1464, in __request
response = self.__transport.request(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/xmlrpc/client.py", line 1166, in request
return self.single_request(host, handler, request_body, verbose)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/xmlrpc/client.py", line 1178, in single_request
http_conn = self.send_request(host, handler, request_body, verbose)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/xmlrpc/client.py", line 1291, in send_request
self.send_content(connection, request_body)
File "/usr/lib64/python3.11/xmlrpc/client.py", line 1321, in send_content
connection.endheaders(request_body)
File "/usr/lib64/python3.11/http/client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib64/python3.11/http/client.py", line 1041, in _send_output
self.send(msg)
File "/usr/lib64/python3.11/http/client.py", line 979, in send
self.connect()
File "/usr/lib64/python3.11/http/client.py", line 1458, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/ssl.py", line 1075, in _create
self.do_handshake()
File "/usr/lib64/python3.11/ssl.py", line 1346, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1002)Which is a satisfying thing to see! It seems my additions are encrypting the connection and I have not inadvertently disabled certificate checking entirely.
I probably need to raise my expectations because I continue to find myself surprised with how easy these things are. To go from nothing to a restricted, socket-activated, and TLS encrypted RPC server with HTTP basic authentication using just the standard library and the facilities provided by systemd in no time has been downright fun. I have previously considered doing TLS termination at the level of reverse proxies the "right way" to design things. TLS termination necessarily has implications for security behind the proxy and I'm now wondering if it isn't worth pursuing ideas like I've demonstrated here, with the encryption baked into the service itself. It imposes some overhead in both development and operation but some of these could be alleviated with better certificate management (like if each service had a certificate signed by a single, internal, trusted CA).
While basic authentication was pleasantly easy to layer over the server I think a more complete example supporting multiple users with a more dynamic credential store could be interesting. Multiple users or client certificates would motivate better logging. It might also be interesting to make this server more observable in general, the current logging is too opaque to be useful. Problems for another time perhaps!