There are 3 ways to control who can have remote access via RDP/Terminal Service
1. Add/Remove users to/from Remote Desktop Users group. This is the recommended way;
2. Define "Allow log on through Terminal Services" in security policy
3. Terminal Services Configuration/Connections/The connection you want to change/Properties/Security/Advanced
Grant or clear permissions here. To enable remote login, a user needs at least the following permissions: Query Information, Logon, and Connect.
By default, administrators can log into DC remotely, admin and RD Users can log into non-DC machine remotely.