Server 2016 RDS via Azure AD Application Proxy end-to-end guide

remote_desktop_blueOne of our priorities for this year was to improve our remote access offering to staff to enable more flexible working whilst outside of college. Office 365 helps greatly and has already improved functionality in many ways but there’s still some legacy applications and classic file shares that need to be provided remotely too. If at all possible we prefer the files not to leave the network so some form of virtual desktop looked the way to go.

After discounting VMware and Citrix offerings on cost grounds the improvements to Microsoft’s RDS offering in Server 2016 seemed to come at a perfect time.

Even more so now we’ve implemented Azure AD Application Proxy (more on that shortly!) We’ve also recently decommissioned some services that freed up a bit of physical hardware resource to “play” with so away we went!

Server installation

The physical hardware for now is running on some reclaimed Dell PowerEdge R610 servers; 64GB RAM, dual CPU and 6 x 15k disks in RAID10. Should be plenty to get us up and running with the RDS roles eventually split across two hosts. For now we’re running on just the one but even that’s plenty to get up and running with.

We installed Server 2016 Core running the Hyper-V role, which was simple enough. The Core role looks to be a tad more polished in Server 2016, although not new the sconfig tool got the main settings entered with fairly minimal fuss.

r610
yes it will go back in the rack once we’re done with it!

Getting the OS to update correctly wasn’t so simple due to Microsoft doing something silly to the update mechanism in the initial release of Windows 10 1607 and its equivalent Server 2016 release. Update status was stuck on “Downloading” showing no signs of progressing. In the end manually installing the latest Cumulative update release from the Microsoft Update Catalog did the trick e.g.

wusa.exe windows10.0-kb3213986-x64_a1f5adacc28b56d7728c92e318d6596d9072aec4.msu /quiet /norestart

Server roles

With Hyper-V up and running the next stage was to install our guests. We went with 3 VMs set up as follows:

  • Connection Broker \ RD Licensing
  • RD Web Access \ RD Gateway
  • RD Session Host

The original plan was to try and embrace the Server Core concept and only install the GUI where absolutely necessary. With that in mind we made the first two servers with Core and only the Session Host with a GUI. More on that soon… (!)

add-roles-wizard
RDS deployment wizard Role Services

Running the deployment through Server Manager on my desktop was easy going, Microsoft have done good work with this and the deployment doesn’t seem too far removed from the 2012 R2 guides I’ve been looking at online. We added each server to the roles as per above, got to the final screen and hit the magic Deploy button then…

"Unable to install RD Web Access role service on server"

Role service... Failed
Deployment... Cancelled

Well that didn’t go to plan! We had a look online, trying to find reasons for the failures and went through some initial troubleshooting to make sure all recent updates were installed and each server’s patches matched exactly, also enabled Powershell remoting…

Enable-PSRemoting -force

…still no joy until we found this little nugget of information…

Ref: https://social.technet.microsoft.com/Forums/Sharepoint/en-US/b5c2bae3-0e3b-4d22-b64d-a51d27f0b0e4/deploying-rds-2012-r2-unable-to-install-rd-web-access-role-service-on-server?forum=winserverTS

So it appears the RD Gateway \ RD Web Access role isn’t supported on Server Core. Of course we wouldn’t want the web-facing part of the deployment running on a server with reduced attack surface would we Microsoft… not impressed!

Ref: https://technet.microsoft.com/en-us/library/jj574158(v=ws.11).aspx

To confirm the hypothesis running Get-WindowsFeature on Server 2016 Core gives this…

server-core-available-rds-roles
Server Core

and on Server 2016 with GUI gives this…

server-gui-available-rds-roles
Server with GUI

Published names & certificate fun and games

After begrudgingly re-installing one of the VMs with a GUI (seemed quicker than trying to convert the Core install) we managed to get past the final Deploy page with 3 success bars 🙂

The first key setting we were asked for was the external FQDN for the RD Gateway, which was added to our ISP-hosted DNS records. We use a wildcard certificate to cover our external facing SSL needs, nothing out the ordinary there and went on to apply it to each of the four roles specified by the RDS Deployment wizard. A Session Collection was created for a test group and pointed at the new Session Host. All looking promising.

The RD Gateway FQDN naming in itself wasn’t a problem but led us to an interesting part of the setup relating to SSL certificates and domains. Once we had the RDS services accessible from outside the network (see below) I fired up my 4G tethering to give it a test.

The connection worked but threw up a certificate warning and it was obvious to see why. Our wildcard certificate is for *.domain.ac.uk but the Connection Broker’s published FQDN is servername.subdomain.domain.ac.uk and therefore isn’t covered.

Fortunately a Powershell script called Set-RDPublishedName exists to change this published name and works a treat! Grab it from https://gallery.technet.microsoft.com/Change-published-FQDN-for-2a029b80

You’ll also need to ensure that you can access the new published name internally, depending on what form your internal domain is vs. your external you may need to do a bit of DNS trickery with zones to get the records you need. More on that can be found at:

Ref: https://msfreaks.wordpress.com/2013/12/09/windows-2012-r2-remote-desktop-services-part-1
Ref: https://msfreaks.wordpress.com/2013/12/23/windows-2012-r2-remote-desktop-services-part-2

set-rdpublishedname
Set-RDPublishedName script in action

External access via Azure AD Application Proxy

We published the RD Gateway and RD Web Access via our new shiny Azure AD Application Proxy for a few reasons…

  • simplicity, no firewall rules or DMZ required
  • security, leverages Azure to provide the secure tunnel
  • SSO, use Kerberos Delegation to sign into RD Web Access as part of the user’s Office 365 login

I followed the excellent guides from Arjan Vroege’s blog for this, in particular the section regarding how to edit the RD Web Access webpage files… nice work Arjan!

Publish your RDS Environment with Azure and AD Proxy – Part 1 – http://www.vroege.biz/?p=2462
Publish your RDS Environment with Azure and AD Proxy – Part 2 – http://www.vroege.biz/?p=2563
Publish your RDS Environment with Azure and AD Proxy – Part 3 – http://www.vroege.biz/?p=2647

As per my previous post on Azure AD Application Proxy & Kerberos delegation use the command below to add the SPN record (replace the FQDN and server name as appropriate)

setspn -s HTTP/servername.subdomain.domain.ac.uk servername

When done the end result is a seamless login to RD Web Access via the Azure AD login page. In our case the link will eventually end up as a button on our Office 365-based Staff Intranet, therefore not requiring any further logins to get to the RDWeb app selection screen.

I particularly wanted to avoid the RDWeb login screen, which I’m amazed in 2017 still requires DIY hacks to avoid the requirement to login with the DOMAIN\username format. Thought Microsoft would’ve improved that in the Server 2016 release but evidently not.

One more gotcha

So having done all the hard work above preparing the login all that was left was to click the Remote Desktop icon and enjoy, right? Wrong.

After running the Set-RDPublishedName script the certificate warning went away and I could see the change to the new wildcard-friendly name, however the connection attempt now failed with the error “Remote Desktop can’t connect to the remote computer *connectionbrokername* for one of these reasons”

remote-desktop-cant-connect
connection failure after changing Published Name

Neither explanation made any sense as the connection was working perfectly fine until changing the Published Name. Indeed changing it back to the original FQDN of the Connection Broker restored service so it had to be something to do with that. After being stumped initially I came back after food (always helps!) then after a bit more research found this very helpful post:

Ref: https://social.technet.microsoft.com/Forums/windowsserver/en-US/4fa952bc-6842-437f-8394-281823b0e7ad/change-published-fqdn-for-2012-r2-rds?forum=winserverTS

It turns out the new FQDN we added when changing the Published Name needs to be added to RDG_RDAllConnectionBrokers Local Computer Group.

This group is used to approve connections in the Resource Authorization Policies (RD-RAP) section of RD Gateway Manager. By default only the server’s domain FQDN is present in the list (as you’d expect) so it appears unless you add the new Published Name in there the connection attempt gets denied.

To add your external published name follow these steps:

  • Server Manager > Tools > Remote Desktop Services > Remote Desktop Gateway Manager
  • expand your RD Gateway server > Policies > Resource Authorization Policies
  • Click Manage Local Computer Groups on the right hand pane
  • Select RDG_RDConnectionBrokers > Properties
  • Click the Network Resources tab
  • type the FQDN of the Published Name you supplied to the Powershell script earlier then click Add
  • OK all the way out then try your connection again

manage-locally-stored-computer-groups
RD Gateway Manager

The example below replaces the real server names with dummy entries but should illustrate the concept. The same scenario applies if your servers exist in a .local Active Directory domain (which will be the top entry) and your external domain is something different (again remember to sort out internal DNS zone entries to suit)

add-external-name-to-rdcbcomputers-group
Manage RDG_RDCBComputers group

Finishing touches

Once all the above is done you should then get a connection, there is one seemingly unavoidable credential prompt due to Microsoft persisting with using an ActiveX control to start the RDP session but perhaps one day they’ll update it (we live in hope). It seems you can use the UPN style format here which is handy as it keeps things consistent. In a way it’s a bit of a security measure so not the end of the world.

Now the connection itself is sorted out all that’s left is to tweak the Session Host to our requirements. This guide gives some nice pointers on locking down the server via GPO:

Ref: http://www.it.ltsoy.com/windows/lock-down-remote-desktop-services-server-2012

We also push out a custom Start Menu using the newer Windows 10 1607 GPO settings along with the Export-StartLayout command. Finally install any programs required, remember to change the mode of the server first:

Ref: https://technet.microsoft.com/en-us/library/ff432698.aspx

change user /install

Then once done

change user /execute

Now enjoy 🙂

rds-screenshot
Connection to Server 2016 RDS Session Based desktop via RD Web Access \ RD Gateway

Advertisements

Azure Active Directory Application Proxy installation and troubleshooting

11225654646_7fc9621cc9_bRecently we decided to migrate away from our legacy reverse-proxy product to something that would integrate better with our AD \ Office 365 systems. I’ve wanted to try out Azure AD Application Proxy for a while since seeing it in beta last year so this seemed a good time to get to grips with it. This post outlines a few gotchas to watch out for and some useful background reading.

Let’s start off with the initial Microsoft documentation available here

https://docs.microsoft.com/en-us/azure/active-directory/active-directory-application-proxy-get-started

Education freebies

Although Microsoft’s recent price hikes haven’t come at a good time for us in education we do get a lot of extras thrown into our Microsoft licensing agreement. One of the lesser-known ones is Azure AD Basic, which is the minimum requirement to use Azure AD Application Proxy – see comparison chart at https://www.microsoft.com/en-cy/cloud-platform/azure-active-directory-features for more info

To get your free licenses you’ll need to get in contact with your EES reseller and they’ll get them added to your tenant in a similar way to Office 365.

Applying the Azure AD Basic license is nice and simple, go to your Azure Management portal at https://manage.windowsazure.com, select your Azure AD directory then assign suitable groups to the license. What’s handy is that if you’re using Azure AD Connect to sync from your on-prem directory any new users will get automatically licensed as they come on board.

Installation

Next step in the documentation list is here:

https://docs.microsoft.com/en-us/azure/active-directory/active-directory-application-proxy-enable

I used two dedicated Server 2012 R2 VMs for our install, the connector will be installed on each so we have failover should it be required at some point. Enabling the Application Proxy in Azure is nothing more than one click in the portal

Now in theory the installation should be straightforward, nothing more than downloading the installer from the link, sign in with admin credentials and job done. However if everything went that smoothly this blog wouldn’t exist (!)

Troubleshooting 403 Forbidden errors

At the end of the installation the wizard helpfully offers to run a troubleshooter to check all is well but in fact all was far from well…

Checking Event Viewer threw up the following errors:

  • Event ID 32012
    The Connector update using the update service failed: ‘The remote server returned an error: (403) Forbidden.’. Check your firewall settings.
  • Event ID 12020
    The Connector was unable to connect to the service due to networking issues. The Connector tried to access the following URL: ‘https://***GUID***.bootstrap.msappproxy.net:8080/’

Outbound firewall settings were already configured to allow all the ports that were asked for in the documentation, proxy was disabled in Connection Settings and the firewall didn’t register any outbound traffic being blocked so what’s going on here? The mystery deepens…

Although the wizard only offers to run the troubleshooter once you can run it again manually by launching it from:

C:\Program Files\Microsoft AAD App Proxy Connector\ConnectorTroubleshooterLauncher.exe

Troubleshooting the troubleshooter

Although there’s a fair bit of documentation in the troubleshooting section on Microsoft’s pages none of it referred to this particular error. Google didn’t have much to go on either but did throw up some useful and detailed slides from the Ignite conference that are well worth a read:

Ref: https://channel9.msdn.com/Events/Ignite/2015/BRK3864
Ref: https://techcommunity.microsoft.com/t5/Microsoft-Ignite-Content/BRK3139-Throw-away-your-DMZ-Azure-Active-Directory-Application/td-p/10675

The second link references another useful document aimed purely at troubleshooting:

Ref: http://aka.ms/proxytshootpaper

Whilst searching I stumbled across an email contact for the Microsoft Azure AD Application Proxy team

aadapfeedback@microsoft.com 

so I dropped them a message with the errors I was encountering. The team replied almost instantly and initially suggested ensuring that the following updates were applied on the server:

https://support.microsoft.com/en-us/kb/2973337
https://support.microsoft.com/en-us/kb/2975719

Proxy proxy proxy!

However still no joy even with everything present as it should be. The next recommendation was to check if I was using a proxy server for outbound connections. We do have one but it’s not used for server VLANs and is the first thing I disable on a new VM build.

However I did get intrigued to check the traffic going out via TCPView… lo and behold there was the proxy server trying to take the outbound connections and failing miserably. It seems that despite everything in the operating system suggesting traffic should be going out directly the Connector was still trying to use the proxy route instead.

Ref: https://blogs.technet.microsoft.com/applicationproxyblog/2016/03/07/working-with-existing-on-prem-proxy-servers-configuration-considerations-for-your-connectors/

The solution is in this document under the section “Bypassing outbound proxies”, which basically involves adding these lines to the .config files for both Connector and Updater services

<system.net>

<defaultProxy enabled="false"></defaultProxy>

</system.net>

Checking Event Viewer and the Azure Portal afterwards showed success, my Connectors were now up and running with nice green icons, much better 🙂

Note: even though this fix resolves the issue the current version of the Troubleshooter doesn’t seem to follow the settings in the .config files and will still report connection failures. The Azure AD Application Proxy team are aware of this and are aiming to have a new version out soon.

Additional considerations

There’s a few other points to bear in mind when you’re completing the configuration of the application proxy. None of them are major issues but good to have everything ready before you start…

Certificates

Once the Connectors are up and running the rest of the process went smoothly, although note you will need a wildcard certificate if you want to publish your applications via a “vanity” URL i.e. your own domain rather than “msappproxy.net”

Using the vanity domain and some DNS CNAME records means that if you use Office 365 SharePoint for your Intranet your internal applications can work from the same URL both inside and outside.

Setting SPNs for Kerberos SSO

Even better, those internal apps can SSO based on the Office 365 initial sign-on for a suitably slick user experience! This does require a bit more configuration with Kerberos delegation but it’s not too bad.

When setting the SPN records I remembered the gotcha from when I worked on Dynamics CRM to type the command in manually… bizarre as it is the same still applies!

Using the -S switch worked well for me:

setspn -s HTTP/yourserver yourserver

Ref: https://blogs.msdn.microsoft.com/saurabh_singh/2009/01/08/new-features-in-setspn-exe-on-windows-server-2008/

Nested groups

Finally, bear in mind if you’re using groups created natively in Azure AD you can’t nest memberships when creating application assignments, which is a shame. As a workaround create any nested ones in your local AD instead and sync them up via Azure AD Connect or just create flat groups in Azure AD if you prefer to work solely up there.

Ref: https://docs.microsoft.com/en-us/azure/active-directory/active-directory-accessmanagement-manage-groups

Application links

You can either publish your application links via your Intranet or users can browse them via the portal (I’ve linked to the new makeover version as it looks much better than the previous one in my opinion)

https://account.activedirectory.windowsazure.com/r#/applications

image credit Rainer Stropek 

Office 365 DirSync experiences: synced OUs and user deletion

DirSync

We experienced an interesting situation the other day with DirSync that doesn’t seem to be documented elsewhere, so thought I’d write it up here for future reference in case anyone hits the same issue…

Our Active Directory is set up to sync users to Office 365 via specific OUs, rather than the entire directory (that includes system users and so on).

In our case we sync staff, students and a spare holding container. This has worked well for us until now with no need for any intervention and users appear in Office 365 once created in AD.

Ref: http://office365support.ca/directory-synchronization-filtering-ous-to-synchronize-to-office-365/

Deletion threshold

We also set up the “prevent accidental deletes” threshold to ensure we had a safeguard in place should a mass deletion event occur. In our case we went for 50 as our limit, which in day-to-day use tends to be about right.

Set-PreventAccidentalDeletes -Enable –ObjectDeletionThreshold 50

Sometimes we have to temporarily raise (then reset) the threshold if a batch of student accounts expire at once but it’s something we don’t need to do that often.

Ref: http://blog.hametbenoit.info/Lists/Posts/Post.aspx?ID=596

Our situation

The trigger for our particular issue was related to a scheduled database job experiencing an error, which led to a batch of users being moved from a synced OU in Active Directory to one that holds expired accounts and as such wasn’t ticked in the Management Agent in DirSync.

As a result on the next run DirSync acted as you’d expect it to and tried to delete the affected accounts from Office 365. Fortunately the PreventAccidentalDeletes threshold kicked in as it should and stopped the action from taking place, then sent a warning to our Network Support email group.

What happens next?

Microsoft have a lot of documentation on setting up the threshold to prevent accidental deletes but don’t expand on the various situations that could cause sync deletions and how to resolve them. For instance, the link below talks only about accounts being deleted from the source Active Directory but this didn’t apply to us; our accounts were still there but had ended up out of scope due to being moved.

Ref: http://social.technet.microsoft.com/wiki/contents/articles/24544.dirsync-how-to-avoid-syncing-accidental-deletes-to-the-cloud-directory.aspx

We resolved the initial issue and moved the affected accounts back into scope via their original OU; however DirSync still wanted to remove the accounts. We ran the standard sync command line…

Start-OnlineCoexistenceSync

…but to no avail. After each run the warning email was still being sent, with the same number of users to be deleted. We also noticed that any new account creation seemed to be stuck in limbo until we either resolved the situation or raised the threshold (second choice wasn’t an option!)

Searching around for suggestions didn’t give much away, although this older article did spark a thought in my head

Ref: https://exitcodezero.wordpress.com/2013/04/29/how-to-force-dirsync-to-perform-full-synchronization/

The point about DirSync running Delta syncs made sense in our context; basically the sync engine was no longer looking for the affected accounts because on the next Delta sync it would assumed they’d been deleted. What I suspected we needed was some sort of Full Sync that would look at all accounts and then decide what to do with each one.

Resolution

At this point although the theory made sense I didn’t want to take any chances so raised a ticket with Microsoft Support to explain the issue. To be fair to Microsoft the speed of response from their support team was excellent and we were soon on a remote session with an engineer.

Initially the suggestion was to re-run the DirSync configuration wizard and start a Full Sync that way. However I didn’t want to do this as we’d made quite a few custom attribute mappings (for GAL separation etc.) and I didn’t want to re-do all of that if at all possible.

The Microsoft support engineer then said he’d trigger a Full Sync another way and opened up PowerShell to run a slight variation on the usual command…

Start-OnlineCoexistenceSync -FullSync

The process took about 10-15 minutes to complete but the next email we received showed the deletion threshold had gone right down, to a level we’d expect. We were able to verify that the accounts left to be deleted were expected (expired accounts) so we then raised the threshold, ran another sync and set it back to 50. New users in the queue were then created as expected and all was calm again in the office 🙂

1427666199_WarningDisclaimer: the information and commands above worked in our situation but are provided for reference only. Given the business-critical nature of mass user account changes in Office 365 I’d always recommend opening a case with Office 365 support before doing anything that could have potentially nasty side-effects!