OIDC/OAuth2 manual flow with curl and Shibboleth

We want to secure our mailservers with two-factor-authentication, but unfortunately, most email clients (e.g. thunderbird) only provide two-factor-authentication flows for defined mail providers like Google or Microsoft (the issue is still open). So I wanted to do a manual OpenID connect / OAuth2 authentication using curl with Shibboleth as an identity provider. Afterwards, a very recently published Thunderbird plugin could help in rolling this out.

1. First, query the /.well-known/openid-configuration endpoint of your identity provider. This should provide a JSON-formatted response describing the available OpenID endpoints, that could look like this:

   {
      "issuer":"https://example.com",
      "authorization_endpoint":"https://example.com/idp/profile/oidc/authorize",
      "registration_endpoint":"https://example.com/idp/profile/oidc/register",
      "token_endpoint":"https://example.com/idp/profile/oidc/token",
      "userinfo_endpoint":"https://example.com/idp/profile/oidc/userinfo",
      "introspection_endpoint":"https://example.com/idp/profile/oauth2/introspection",
      "revocation_endpoint":"https://example.com/idp/profile/oauth2/revocation",
      "jwks_uri":"https://example.com/idp/profile/oidc/keyset",
      "response_types_supported":[...],
      "subject_types_supported":[...],
      "grant_types_supported":[...],
      ...
   }
   

   Take note of the authorization_endpoint and the token_endpoint.

2. Obtain a client id and client secret for the Shibboleth identity provider, for example by registering a test application. Make sure to set the redirect URI to something you control, either a nonexistent webserver or simply localhost.

3. Open the authorization endpoint URL with valid parameters for client_id, redirect_uri, response_type=code and scope=openid in a webbrowser. It may look like this:

    https://example.com/idp/profile/oidc/authorize?client_id=[...]&redirect_uri=https://localhost&response_type=code&scope=openid
   

4. Complete the Shibboleth login flow in your webbrowser, using username, password and potential second factor of a valid user account.

5. After logging in, Shibboleth will generate an authorization code and issue a redirect to the specified redirect URI, with the authorization code passed as an URL parameter. For example:

   https://localhost/?code=AAdzZWNy[...]DwCQ4tB7WbCTdWVQ
   

6. Now, issue a curl request to the token endpoint from step one, passing valid client_id and client_secret, grant_type=authorization_code and the authorization code from the redirect URL as code in URL parameters. Make sure the request is a POST request with the Content-Type: application/x-www-form-urlencoded header set. This can be achieved by passing --data-urlencode to curl, e.g. like this:

   curl -H 'Content-Type: application/x-www-form-urlencoded' "$token_endpoint?client_id=[...]&client_secret=[...]&grant_type=code" --data-urlencode "code=AAdzZWNy[...]DwCQ4tB7WbCTdWVQ
   

   Make sure to issue this request within seconds after receiving the authorization code, as the authorization code seems to expire very fast. It may help to prepare the curl request before completing the login flow.

7. Shibboleth should return a token in a JSON-formatted reply, e.g. like this:

   {"access_token":"eyJr[...]2In0.eyJz[...]yIn0.bP2a[...]Ls5X","scope":"openid","id_token":"eyJr[...]2In0.eyJh[...]gifQ.uOgA[...]Kzvf","token_type":"Bearer","expires_in":600}
   

   The access_token can be used for logging in with services, e.g. a dovecot mail server as described in the corresponding open-xchange manual .

Deciphering the token

The token provided in access_token is a JSON Web Token (JWT) that consists by design of three parts separated by dots. The first part represents the JWT header, giving metadata on the format and signature of the JWT. The second part represents the JWT body, containing the actual information. The third part holds the JWT signature.

The first two parts are JSON-strings in base64-encoding. When decoded, they look like this:

{"kid":"defaultRSASign","typ":"at+jwt","alg":"RS256"}
{"sub":"user","iss":"https://example.com","for_op":"AAdz[...]HkeQ","client_id":"[...]","sid":"_3a28[...]","aud":"https://example.com","root_jti":"_6521[...]","auth_time":1768921036,"scope":"openid email","exp":1768921642,"iat":1768921042,"jti":"_1d925[...]"}

Automating the procedure using python and http.server

To automate the token reception and deciphering, a small python script can help:

from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse, parse_qs
import time, ssl, json, requests, base64

hostName = "localhost"
serverPort = 443

class MyServer(BaseHTTPRequestHandler):
    def __init__(self, *args, discovery_endpoint="https://example.com/.well-known/openid-configuration", **kwargs):
        self.discovery_endpoint = discovery_endpoint
        discovery_result = requests.get(self.discovery_endpoint).json()
        self.token_endpoint = discovery_result["token_endpoint"]
        self.introspection_endpoint = discovery_result["introspection_endpoint"]
        self.userinfo_endpoint = discovery_result["userinfo_endpoint"]
        super().__init__(*args, **kwargs)
    
    def do_GET(self):
        self.send_response(200)
        self.send_header("Content-type", "text/html")
        self.end_headers()
        self.wfile.write(bytes("<html><head><title>OAuth2 Access Token converter</title></head>", "utf-8"))
        self.wfile.write(bytes("<body>", "utf-8"))
        self.wfile.write(bytes("<h1>OAuth2 Access Token converter</h1>", "utf-8"))
        self.wfile.write(bytes("<p><b>Request:</b> %s</p>" % self.path, "utf-8"))
        
        try:
            query_components = parse_qs(urlparse(self.path).query)
            code = query_components["code"][0]
            username = "username@example.com"
            client_id = "[...]"
            client_secret = "[...]"
            headers = {
                "Content-Type": "application/x-www-form-urlencoded",
            }
            params = {
                "client_id": client_id,
                "client_secret": client_secret,
                "grant_type": "authorization_code",
                "scope": "openid profile email openid",
                "code": code,
            }
            response = requests.post(self.token_endpoint, headers=headers, params=params)
            access_token = response.json()["access_token"]
            
            self.wfile.write(bytes("<p><b>Contains authorization Code:</b> %s</p>" % code, "utf-8"))
            self.wfile.write(bytes("<p><b>Sending request to:</b> %s</p>" % self.token_endpoint, "utf-8"))
            self.wfile.write(bytes("<p><b>With headers:</b> %s</p>" % headers, "utf-8"))
            self.wfile.write(bytes("<p><b>And parameters:</b> %s</p>" % params, "utf-8"))
            self.wfile.write(bytes("<p><b>Response of /token:</b> %s</p>" % response.text, "utf-8"))
            self.wfile.write(bytes("<p><b>Resulting Access Token:</b> %s</p>" % access_token, "utf-8"))
            
            token_parts = access_token.split(".")
            if len(token_parts) != 3:
                self.wfile.write(bytes("<p><b>Access Token has invalid number of parts:</b> %s</p>" % len(token_parts, "utf-8")))
            else:
                (token_header, token_payload, token_signature) = token_parts
                
                def token_b64decode(b64str):
                    # Add padding if necessary
                    padding = len(b64str) % 4
                    if padding:
                        b64str += '=' * (4 - padding)
                    return base64.urlsafe_b64decode(b64str)
                
                self.wfile.write(bytes("<p><b>Token header:</b> %s</p>" % token_b64decode(token_header), "utf-8"))
                self.wfile.write(bytes("<p><b>Token payload:</b> %s</p>" % token_b64decode(token_payload), "utf-8"))
                self.wfile.write(bytes("<p><b>Token signature:</b> %s</p>" % token_b64decode(token_signature), "utf-8"))
            
            headers = {
                "Content-Type": "application/x-www-form-urlencoded",
            }
            params = {
                "client_id": client_id,
                "client_secret": client_secret,
            }
            data = {
                "token": access_token,
            }
            self.wfile.write(bytes("<p><b>Introspection params:</b> %s</p>" % params, "utf-8"))
            
            response = requests.post(self.introspection_endpoint, headers=headers, params=params, data=data)
            self.wfile.write(bytes("<p><b>Introspection result:</b> %s</p>" % response.text, "utf-8"))
            
            headers = {
                "Content-Type": "application/x-www-form-urlencoded",
                "Authorization": f"Bearer {access_token}",
            }
            params = {
                "client_id": client_id,
                "client_secret": client_secret,
            }
            self.wfile.write(bytes("<p><b>Userinfo params:</b> %s</p>" % params, "utf-8"))
            
            response = requests.get(self.userinfo_endpoint, headers=headers, params=params)
            self.wfile.write(bytes("<p><b>Userinfo result:</b> %s</p>" % response.text, "utf-8"))
            
            xoauthstring = f"user={username}\001auth=Bearer {access_token}\001\001"
            
            self.wfile.write(bytes("<p><b>Converting to XOAUTH2 string:</b> %s</p>" % xoauthstring, "utf-8"))
            
            base64result = base64.b64encode(str.encode(xoauthstring))
            
            self.wfile.write(bytes("<p><b>Resulting base64 string:</b> %s</p>" % base64result, "utf-8"))
        except Exception as ae:
            print(ae)
        
        
        self.wfile.write(bytes("</body></html>", "utf-8"))

if __name__ == "__main__":        
    webServer = HTTPServer((hostName, serverPort), MyServer)
    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    ssl_context.load_cert_chain(certfile='cert.pem', keyfile='key.pem')
    webServer.socket = ssl_context.wrap_socket(webServer.socket, server_side=True)
    print("Server started https://%s:%s" % (hostName, serverPort))

    try:
        webServer.serve_forever()
    except KeyboardInterrupt:
        pass

    webServer.server_close()
    print("Server stopped.")

After generating a self signed certificate using openssl:

openssl genrsa -out key.pem 2048
openssl req -new -x509 -days 365 -key key.pem -out cert.pem

the script can be executed using sudo python script.py (sudo is required as the script will listen on port 443). If the redirect URI is set to https://localhost, the script will be able to receive the authorization code when the browser is redirected from Shibboleth, issue a token at the identity provider, and display the token’s content as decoded from base64. It will also generate and base64 encode the required login string that can be used to do a XOAUTH2 authentication in dovecot via telnet. An example looks like this: