Lync Certificate Provisioning Service Deeper Dive

Welcome back, this continues on from my previous LyncDiscover and Auto Discovery Deeper Dive article. I would suggest going back and reading that for some background on how we got up to here. But shouldn’t be required to follow this, the topics covered here are widely used across the web on many platforms and applications, and are not specific to Lync nor Microsoft. With a little bit of explaining (another long article), I aim to describe a little bit about these technologies, and why Lync uses them.

Previously we saw how to use LyncDiscover and Auto Discovery to get a Lync Pool FQDN for a particular user, and which ports to connect to, as well as the Web Service URLs. This required that we first authenticated with NTLM, to obtain either an OAuth Token, or a WebServiceTicket. Which we used to get the Auto Discover XML.

That seems like an awful lot of different ways just to identify yourself. So lets take a step back, and look at them…

NTLM is used to authenticate a connection using a hash of your Active Directory password, this is a challenge-response with 3-4 messages in the handshake alone, which needs to occur for every TCP connection, and also requires a Domain Controller to be online and accessible to verify your credentials, therefore increased network traffic, and latency to establish a connection.

OAuth is becoming popular with web based applications, and how they interact to provide authorisation to 3rd parties, used by Twitter, Google, Yahoo, and Facebook etc. OAuth is fairly new to Microsoft Products, and seems to be aimed at UCWA / Web based applications. And because traffic is encrypted end to end using TLS, it also makes it easy for the application to a send plain-text username and password, which the server then processes against Active Directory; and if successful, issues you with a “bearer token” that allows you to re-assert your authorisation without the handshake nor providing your Active Directory credentials every time.

WebTicket uses much more flexible and standards based SAML wrapped in SOAP, and is a fully fledged claims based authentication mechanism, and can use a wide range of encryption, and hashing algorithms to suit. Like OAuth, once you have authenticated you simply use the provided token / ticket on subsequent connections to assert your authorisation.

What’s the difference

This is going to be a rather a simplistic view, but the advantages that OAuth and WebTicket have (being AuthZ) over NTLM (which is AuthN) is, when using NTLM, you are authenticating yourself to a resource every time. For example, when making a complaint to your broadband provider, your account number and password are checked against their database, which is how they know it’s YOU and are subsequently authorised to talk about cancelling. But what if it doesn’t really matter WHO you are, just that you are allowed to do something. If you’ve dealt with ADFS, or any sort of Federation (i.e. Shibboleth, or older OpenAthens) you should be familiar with what I’m trying to describe. It’s like showing your driving licence to a policeman, he trusts the DVLA, therefore trusts their ‘assertion’ of your identity and that you are authorised to drive.

That all might sound pointless and over the top, but when you consider the scale that Lync can reach, with multiple pools of servers carrying real-time data, it actually makes it much easier to just ‘show your pass on the door’ instead of providing personal identifiable information and have your name checked against a list, to request entry every time you walk in.

Recap

NTLM allows “me” to authenticate my identity for a specific connection. The OAuth Token, or WebTicket obtained from that connection authorises the application to act as me in a certain capacity on subsequent connections, until access expires.

Limitations

Tokens / Tickets should generally be short-lived, as mere proof-of-possession is all that’s required, the longer this piece of information exists the bigger the chance of it being intercepted, or stolen somehow.

The default duration for this is configured in an XML configuration file in the following location… (under both Int and Ext sub-folders)

Inside web.config, “<configuration> » <Microsoft.Rtc.Internal.WebTicketService> » <WebTicketTimerConfig>”, the default values are:

PublicExpireInterval 15 minutes
PublicRefreshInterval 10 minutes
PrivateExpireInterval 8 hours
PrivateRefreshInterval 6 hours

So, from outside, your WebTicket will expire after 15 minutes, and can only be refreshed after 10 minutes.

I’ve not been able to figure out where the OAuth expiry is configured, when you obtain a bearer token the JSON it responds with has an “expires_in” value which seems to be a slightly random figure of roughly 7-8 hours, or thereabouts. RFC6749 specifies a default of 3600 (1-Hour), so there is something specific to Lync going on to override that.

Lync Certificate Provisioning Service

Hopefully there are still a few readers left. For those of you how have stuck around, there’s plenty more to come so lets get on with the story. Back to Lync, and yet another way to provide auth.

Here’s a chance to use something more secure, something that isn’t simply “provided” to the application after supplying your credentials that could potentially be ‘sniffed’ and stolen. That is where Certificate Provisioning Service comes in. Using X509 Certificates as an asymmetric encryption algorithm with a private / public key pair, the private key is generated locally by the client and never transmitted, therefore a much higher assurance that the party encrypting and transmitting data is the same who originally requested the certificate. And as a reflection of the increased assurance of X509 over the other technologies, Lync has a default expiry of 180 days, but allows the client to request it’s own expiry, anywhere from 8 hours, up to a year.

* IMPORTANT – Due to the de-coupling of the Active Directory account, and the certificate issued to allow clients to act on it’s behalf for potentially up to 1 year. It’s important to know that disabling an account in AD, does NOT mean they will be denied from logging into Lync and making a call. But X509 Certificates offer a really easy way to manage this, called Certificate Revocation. Which means whether the AD Account is disabled or not, Lync can check the validity (including revocation as well as expiry) of the client certificate – see Revoke-CsClientCertificate.

Using the following PowerShell cmdlet you can check the minimum, default, and maximum duration that a certificate can be used for…

These default values are good enough for the majority of deployments, there it not usually any reason to change those. Unless your Security Manager has policies in place that limit the duration of client certificates.

You can also see the minimum and maximum key length requirement for client certificates…

And you can use the corresponding  Set-CsWebServicesConfiguration to configure those values.

But be careful, if you increase the key size up to 2048 bits, but you have older clients that request 1024, then the request will be rejected with the following Error.

Lync 2013 Client uses a 2048bit key, if you set higher than that on the server, the Lync Client will have it’s certificate request rejected, but unless you look for it, it’s not obvious, as Lync will resort to using NTLM Authentication. This essentially has the unwanted side effect of disabling Client Certificate authentication, so be careful what you set.

The same applies for altering the validity period, if a client specifically asks for a value that is outside your limits, the certificate request may fail with a similar “InvalidValidityPeriod” error. So testing is important before you cripple your production environment. Although MS-OCAUTHWS defines that as a response type, and Lync 2013 Client includes a validity period in the certificate request of 4320 hours, if the server is configured with a lower maximum, the certificate still gets signed, and returned with a shorter validity period that matches the server’s configuration.

What we need to understand first

Requesting an X509 certificate essentially is just a SOAP Message with a GetAndPublishCert body, just like how we requested the WebTicket in the first place. But this time in the header of this SOAP Message we are presenting a copy of our WebTicket for auth (the SAML Assertion that we obtained in my previous post). And furthermore this also includes an XML Digital Signature (XMLDSIG), to ensure the message is not tampered with, or altered in any way. There are several different ways XMLDSIG can be used, i.e. to sign an external file referenced by a URL, to sign an entire XML Document and add the signature inside that file (obviously the signature element itself isn’t included in the hash algorithm), or just a reference to a one or more elements only.

The Certificate Provisioning Service only requires that a signature is created for a single element. That element is a time-stamp (Creation Time + Expiry Time 5 Minutes later). And has an ID of “#_0”.

Because the signature is to be based on plain text XML, it leaves it open to different ways to display the same XML (i.e. case, tab size, white space, line breaks etc), all of that would change the signature. To allow for this, the XML element undergoes canonicalisation (known as C14N) – it basically describes how to represent XML in a particular way. For example the order attributes are listed within an element, linebreaks are 0x0A, unnecessary white-spaces, and superfluous namespace declarations are removed. After that, no matter which program creates the time-stamp, of any platform, in any way… we will always end up with the exact same ‘string’ of XML which contains our time-stamp, and therefore the same hash.

The signature is created by using a symmetric key hashing algorithm. And we derive the key from two important bits of information, the first is the <BinarySecret> we sent as <Entropy> used for the original WebTicket request – (which was a randomly generated AES256 key), and the second is <BinarySecret> in the <Entropy> of the XML when we got our SAML Assertion. After Base64 decoding that, we can use PSHA1 to compute the symmetric key derived from entropies provided by both parties; therefore we both have the same key to encrypt (sign), and decrypt (verify) messages.

Obtaining a certificate

The first step is to create a certificate request, I used the quite well known “Bouncy Castle Crypto API” for C Sharp which made that process easy. Once I had generated the key pair, and had the Base64 encoded certificate request, I will come back to this later on.

The SOAP Message we need to send has the following structure…

MessageID represents this conversation with a unique GUID. The response from the server will need to match up with this ID, so both parties know it’s the same conversation.

Specifying the web service URL and WebTicket_Proof method.

The Security information being passed to the Certificate Provisioning Server contains a simple time-stamp element. This is to ensure the request expires after 5 minutes. This is the element that is signed using XML Digital Signature.

This is the Web Service Ticket we obtained in the previous post.

This <SignedInfo> element is the actual XML digital signature (XMLDSIG or just DSIG). You can see the canonicalisation method specified, as well as the signature method used to sign the combined references. The <Reference> element points to a URI of #_0, this you can see further up is the ID of the time-stamp element. Each reference specifies it’s own canonicalisation method, and hashing algorithm (C14N, and sha256). This element can be repeated for each reference. And then the following <SignatureValue> is computed against the combined references.. But we only have one here.

Here the <SignatureValue> is the HMAC-SHA256 symmetric key hashing algorithm, using the symmetric key we computed earlier.

So we have a SHA256 hash <DigestValue> of each XML Element we referenced (the time-stamp), then our key is used to create a hash-based authentication code (HMAC using AES256 with our symmetric key) for the XMLDSIG. We now have a time-stamp that cannot be changed, and that the server knows was provided and signed by me. Because otherwise,  it would break the verification of the hash, which itself is signed by our key. The remote party uses the symmetric key to decrypt and verify that  the <DigestValues> of each <Reference> matches with the canonicalised XML of the element that we provided it.

Trust me, that was not fun to learn and fully appreciate what’s going on there, it’s also been difficult to relate that back in this post, where it’s not even the major topic. So I appreciate I may have not explained that part the best, more information will be provided in a set of links at the end.

Now we get to the meat.. the body of the SOAP Message, which is…

Lots of that is taken up with namespace declarations, request type, and encoding type URIs; it just describes the schema of the message, and what format the data we are sending is.

The DeviceID attribute is a unique GUID to identify the device requesting the certificate, therefore the same device should never normally request multiple certificates to be valid at the same time, if the same DeviceID is used, the previous certificate will be revoked and a new one issued.

And the <BinarySecurityToken> contains the actual Base64 encoded X509 Certificate Request. (i.e. the contents of a .csr or .req file)

We need to send this in a HTTP POST to the Certificate Provisioning Service URL we obtained in the Auto Discover XML previously.

e.g. https://lyncweb.sipdomain.co.uk/CertProv/CertProvisioningService.svc/WebTicket_Proof

We also need to insert the following SOAPAction HTTP Header

Hopefully we would then receive an ‘Issued’ message containing our signed certificate back.

Just in case we don’t, here’s a couple of common errors I encountered.

InvalidSecurityToken

This is returned when there is a problem with the time-stamp element, i.e. it’s missing, incorrect, or it has expired.

InvalidSecurity

This is returned when there is a problem with the XMLDSIG, normally because the symmetric key used to sign it differs.

Our X509 Certificate

After a successful request, we receive our certificate stuffed inside the following SOAP Message.

Notice the time-stamp in the response here isn’t signed. We went to all that trouble, I’m a little upset that Lync didn’t return the effort. 🙁

The <BinarySecretToken> inside <RequestedSecurityToken> is actually our signed certificate returned to us… Combine that with our private key generated by Bouncy Castle earlier up, and we’ve got ourself a client certificate signed by the Lync Server.

Lync Tester (Lyncer)

The whole point of this little adventure, was to write a Lync Connectivity Tester program of my own (as the best thing around was the NextHop version ‘Lync Connectivity Analyser’, but that hadn’t worked on Windows 8.1 for quite sometime). It started as a dive into the LyncDiscover and Auto Discovery services has now turned into a massive journey into the protocols, authentication, and authorisation used when talking to Lync.

The next step will be to use this certificate to authenticate to the SIP Service on the Access Edge or the Front End, and register with it. Then I think it’ll be in a place suitable to test from initial DNS Lookup, to client registration, providing a copy of all the details along the way, intended for the type of person that’s likes a little bit more detail than just seeing “successful” in a list.

Feel free to leave comments, especially if you feel I could add some further clarification / explanations on the topics covered, and I’ll do my best to ‘flesh it out’. There is obviously a lot more to these standards, protocols, and signatures than is really needed here, or in general.

Thanks for reading. I hope you’ve found what you were looking for. I certainly did.

Resources

MSDN – [MS-OCAUTHWS]: OC Authentication Web Service Protocol

W3C – XML Signature Syntax and Processing (XML DSIG)

MSDN – Understanding WS-Security

OASIS – WS-Trust 1.3

Tweet about this on TwitterShare on LinkedInShare on Facebook
Pin on PinterestShare on Google+Digg thisShare on RedditShare on StumbleUponEmail this to someone

About Graham Cropley

Working as a Senior Consultant for Skype for Business, Exchange, and Office 365.

5 Comments

  1. Robson de Carvalho

    Amazing work bro… congrats and carry on!

  2. Excellent post. Thanks for the concise write up!

  3. nice piece of work. Its hard to remember this stuff even after you learn it. it is nice to have a piece to refer back to. Its a beautiful article.

    Louis

  4. What happens after the webticket is given to the client and your password changes in AD? You still have 170 days left of your initial 180 days before the webticket expires so I assume you can still connect to Lync even though you have a different AD password? You don’t have to enter your new password as long as the app is open? Is the webticket cached somewhere even after a client shutdown or workstation reboot?

    • The webticket is only valid for 8 hours, but it is used to obtain a certificate (the 180 days you mentioned). That certificate is located in your personal certificate store for that user, that’s how it persists between logons etc. And yes, that is also correct changing AD password doesn’t affect a user that’s already logged on… but it’s important to note that even if you disable the AD Account of a user, they can still sign in to Lync/Skype for Business using the certificate – take a look at Revoke-CsClientCertificate (https://technet.microsoft.com/en-us/library/gg425748.aspx)

Leave a Reply

Your email address will not be published. Required fields are marked *