:Troubleshoot remote connection failures to a Windows instance

Use the self-service troubleshooting tool

The Alibaba Cloud self-service troubleshooting tool helps you quickly check security group configurations, the instance firewall, and the listener status of common application ports. The tool provides a detailed diagnostic report.

If the self-service troubleshooting tool cannot identify the issue, follow these steps to troubleshoot the issue manually.

Manual troubleshooting

Follow these steps to check the status of the ECS instance. Then, use Cloud Assistant to send commands to the Windows instance or log on to the instance using VNC.

Step 1: Use Alibaba Cloud Workbench to test the remote logon

You can use the Workbench tool provided by Alibaba Cloud to log on remotely. If the remote logon fails, Workbench returns a specific error message and a solution. Follow these steps to perform the test:

1. Go to ECS console - Instance.

2. In the top navigation bar, select the region and resource group of the resource that you want to manage.

3. On the Instances page, find the target instance, and in the Actions column, click Remote Connection.

4. In the Remote Connection dialog box, under Connect With Workbench, click Log On Now.

5. Workbench automatically fills in the basic information required to log on to the target instance. Confirm that the information is correct and enter the username and authentication credentials. Then, take the appropriate action based on the result:

   • If you still cannot log on, Workbench returns an error message and a solution. Follow the on-screen instructions to resolve the issue and then try to connect again. You can connect to the instance using VNC to resolve common issues that occur when you use Workbench.

   • If you can log on to the instance using Workbench but cannot log on remotely from your local server, the remote connection port and service are working as expected. In this case, you must troubleshoot the issue on your local client.

Step 2: Check for blackhole filtering notifications

Check whether you have received a blackhole filtering notification for the instance. During blackhole filtering, the instance cannot access the Internet. For more information, see Alibaba Cloud blackhole filtering policy.

Step 3: Check ports and security groups

Check whether the security group rules are blocking the connection. Follow these steps:

1. Go to ECS console - Instance.

2. In the top navigation bar, select the region and resource group of the resource that you want to manage.

3. On the Instances page, click the instance ID.

4. Click the Security Groups tab. Find the security group you want to manage in the Security Group List and click Manage Rules in the Actions column.

5. Select the direction for the security group rule.

6. On the Security Group Details tab, you can use one of the following methods to add a security group rule.

   • Method 1: Quickly add a security group rule

     • Select RDP Connection to a Windows Instance

   • Method 2: Manually add a security group rule

     • Action: Allow

     • Priority: 1 (A lower value indicates a higher priority, with 1 being the highest priority.)

     • Protocol: Custom TCP

     • Source > IPv4: 0.0.0.0/0 (represents all IP addresses)

     • Destination (This Instance) > Port : Set to the RDP port. The default is 3389.

7. Connect to the remote desktop in the IP:Port format.

   

8. Run the following command to test the port and determine whether it is working correctly.

   telnet <IP> <Port>

   Note

   • <IP> refers to the IP address of the Windows instance.

   • <Port> refers to the RDP port number of the Windows instance.

   For example, after you run the telnet 192.168.0.1 4389 command, the following command output is returned.

   Trying 192.168.0.1 ...
   Connected to 192.168.0.1  4389.
   Escape character is '^]'

   If the port test fails, see Check port availability when the ping command is successful but the port is unreachable for troubleshooting.

Step 4: Check the firewall configuration

1. Connect to the Windows instance using VNC.

2. In the menu bar, select Start > Control Panel.

3. Set View By to Small Icons. Then, click Windows Firewall.

   

4. In the Windows Firewall window, click Advanced Settings.

5. Enable the firewall configuration.

   1. In the Windows Firewall With Advanced Security window, click Windows Firewall Properties.

      

   2. Select On (recommended) and click Apply.

      We recommend enabling the firewall on the Domain Profile, Private Profile, and Public Profile tabs.

      

6. In the Windows Firewall With Advanced Security window, click Inbound Rules. In the right pane, scroll to the bottom. Right-click Remote Desktop - User Mode (TCP-In) and select Enable Rule.

Step 5: Check the Remote Desktop service

Check whether Remote Desktop Service is enabled on the Windows server. Follow these steps:

1. Connect to a Windows instance using VNC.

2. Right-click the Start menu and click System.

3. In the System window, click Remote Settings.

   

4. In the Remote Desktop area, you can select Allow Remote Connections To This Computer, and then click OK.

   

5. Start the Remote Desktop Services service.

   In the Start menu, select Administrative Tools > Component Services > Services (Local). In the right pane, find the Remote Desktop Services service and check its status. If the service is not running, start it.

   

6. Load the drivers and services on which Remote Desktop Service depends.

   For security purposes, some key services on which Remote Desktop Service depends are sometimes disabled by mistake. This can cause Remote Desktop Service to stop working correctly. To resolve this, perform the following check.

   1. Right-click the Start menu, click Run, enter msconfig, and click OK.

      

   2. In the System Configuration dialog box, on the General tab, select Normal Startup, and then click OK.

      

   3. Restart the ECS instance.

Step 6: Check the remote terminal service configuration

The remote desktop of a Windows instance may be unreachable because of incorrect configuration of the remote terminal service.

Step 7: Check the network

If you cannot connect to a Windows instance remotely, first check whether the network is working correctly.

1. Use computers in other network environments, such as different network segments or from different carriers, to test the connection. This helps determine whether the issue is with the on-premises network or the server.

   • If the issue is with your on-premises network or carrier, contact your local IT staff or the carrier to resolve it.

   • If the network interface card (NIC) or NIC driver is not working correctly, you may need to update the NIC driver. Follow these steps:

     Solution to connection failures caused by a faulty network interface card or driver on a Windows system

     Note

     This operation uses Windows Server 2016 as an example. The UI may differ on other versions. Adjust the steps accordingly for your operating system version.

     1. Connect to a Windows instance using VNC.

     2. In the notification area, right-click the icon and select Open Network and Sharing Center.

     3. Click Change Adapter Settings to check whether the NIC is enabled.

        • If the network interface card is disabled, right-click it and select Enable. Verify that the remote connection to the Windows instance is restored.

        • If the NIC is enabled but still unavailable, proceed to the next step.

     4. Open the Run window, enter regedit, and click OK.

     5. In Registry Editor, navigate to HKEY_LOCAL_MACHINE > SYSTEM > CurrentControlSet > Control > Class > {4d36e972-e325-11ce-bfc1-08002be10318} and verify that the following entries exist in the right pane. If the entries do not exist, right-click a blank area and select New to add them.

        Important

        After you change the information in the Registry Editor, you must restart the system for the changes to take effect.

        • Name: Installer32

        • Type: REG_SZ

        • Data: NetCfgx.dll,NetClassInstaller

     6. Open the Run window, enter devmgmt.msc, and click OK.

     7. In Device Manager, under Network Adapters, right-click the network adapter and select Update Driver Software....

     8. In the dialog box that appears, click Search Automatically For Updated Driver Software. When the update is complete, click Close.

     9. Confirm that the remote connection to the Windows instance is restored.

2. Run the ping command on your local client to test the network connectivity to the instance.

   • If a network issue occurs, see Use a packet capture tool to capture network packets for troubleshooting.

   • If ping packets are lost or the ping fails, use MTR to analyze network paths for troubleshooting.

   • If intermittent packet loss occurs and the network of your ECS instance is unstable, see Link interruption to resolve the issue.

3. If you receive a General Failure error when you ping a client from your instance, see A "General failure" error is reported when you ping a public IP address from a Windows instance to resolve the issue.

Step 8: Check the CPU load, bandwidth, and memory usage

A remote connection to a Windows instance may be unreachable because of high CPU load, insufficient bandwidth, or insufficient memory.

1. Select the appropriate operation based on the CPU load.

   • If the CPU load is not high, proceed to Step 2 to continue troubleshooting.

   • If the CPU load is high, resolve the issue as described in this step.

     • Log on to the instance from the terminal on the Instance Details page and check whether Windows Update is running in the background. A high CPU load is expected if Windows Update is running. Wait for the update to complete.

     • If the applications that are hosted on an instance perform many disk read/write operations, initiate many network requests, or generate compute-intensive workloads, a high CPU load is expected. In this case, you can upgrade the instance type to resolve resource bottlenecks.

       Note

       For more information about how to resolve high CPU loads, see What do I do if a Windows instance has high CPU utilization?.

2. Troubleshooting insufficient public bandwidth.

   A remote connection may fail due to insufficient public bandwidth. Follow these steps to troubleshoot the issue.

   1. Go to ECS console - Instance.

   2. In the top navigation bar, select the region and resource group of the resource that you want to manage.

   3. On the Instances page, click the instance ID. On the Instance Details tab, you can view the Public Bandwidth in the Configuration Information section.

      If the public bandwidth is 0 Mbps, the instance was created without public bandwidth. You can resolve this issue by upgrading the public bandwidth.

3. Check for insufficient memory.

   After you connect to a Windows instance remotely, the desktop may not display correctly and the connection may close without an error message. This can be caused by insufficient server memory. Follow these steps to check the memory usage.

   1. Connect to a Windows instance using VNC.

   2. Go to Start > Administrative Tools > Event Viewer and check for warning logs that indicate insufficient memory.

Step 9: Check the system security policy settings

Check whether any security policies on the Windows server are blocking remote desktop connections. Follow these steps.

1. Connect to a Windows instance using VNC.

2. Select Start > Control Panel > Administrative Tools, and then double-click Local Security Policy.

3. In the Local Security Policy window, click IP Security Policies On Local Computer. The next step depends on whether a security policy already exists.

   1. If a relevant security policy exists, delete or edit it.

      • To delete the security policy, right-click it and select Delete. In the dialog box that appears, click Yes.

      • Double-click the IP security policy to open it, reconfigure it to allow remote desktop connections, and then try to connect again using Remote Desktop.

   2. If a relevant security policy does not exist, go to Step 9: Check the system security policy settings.

Step 10: Check the antivirus software

Remote connection failures may be caused by third-party antivirus software settings. Use the following methods to resolve this issue. This section provides two case studies of how SafeDog configuration can cause remote access to fail.

• If antivirus software is running in the background, connect to the instance using VNC to upgrade the antivirus software to the latest version or uninstall it. For more information about how to connect to an ECS instance using VNC, see Methods for connecting to an ECS instance.

• Use a commercial version of antivirus software or the free Microsoft Safety Scanner to scan for and remove viruses in safe mode. For more information about Security Scanner, see Security Scanner.

Step 11: Abnormal Windows registry configuration

Incorrect configurations in the Windows registry may block RDP connections. Follow these steps to fix the issue.

1. Connect to an instance using VNC.

2. In the Run dialog box, enter regedit and click OK to open the Registry Editor.

   

3. In Registry Editor, modify the following parameter configurations.

   • Set the fEnableWinStation parameter in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp to 1.

   • Set the fDenyTSConnections parameter in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server to 0.

Step 12: The Windows RDP self-signed certificate has expired

An expired RDP self-signed certificate may cause remote logon failures. Follow these steps to fix the issue.

1. Connect to a Windows instance using VNC.

2. Run Windows PowerShell as an administrator.

3. In the Windows PowerShell window, run the following command to check whether the current certificate has expired.

   Get-Item 'Cert:\LocalMachine\Remote Desktop\*' | Select-Object NotAfter

4. If the certificate has expired, run the following command to delete the self-signed certificate and restart the TermService service.

   Remove-Item -Path 'Cert:\LocalMachine\Remote Desktop\*' -Force -ErrorAction SilentlyContinue
   Restart-Service TermService -Force

   After the TermService service is restarted, the system automatically generates a new self-signed certificate.

5. Run the following command to confirm that the new self-signed certificate's timestamp has been updated.

   Get-Item 'Cert:\LocalMachine\Remote Desktop\*' | Select-Object NotAfter

   Note

   The default validity period of an RDP self-signed certificate is six months.

A specific error message is returned

• A protocol error occurs when you remotely connect to a Windows ECS instance

• Authorization-related:

  • Enabling password-protected sharing causes the 'Your credentials did not work' error during remote access to a Windows system

  • The "The connection was denied because the user account is not authorized for remote login" error occurs when you connect to an ECS instance using Remote Desktop

  • Solution for an unspecified Remote Desktop license server in a Windows instance

  • What do I do if the error message that indicates lack of Terminal Server User Access permissions appears when connecting to a Windows instance using Remote Desktop?

• Connection count-related:

  • What do I do if the error message "The terminal server has exceeded the maximum number of allowed connections" appears when connecting to a Windows instance using Remote Desktop?

References

• What do I do if I cannot connect to a Linux instance?

• VNC connection issues

• SSH connection issues

• RDP connection issues