How to fix the four biggest problems with failed VPN connections

Virtual private networks have risen from obscurity to become the frequently preferred method of linking private networks. Although VPNs became popular because they enabled using the Internet to secure network connections, thereby eliminating the need for expensive dedicated circuits, VPN adoption skyrocketed because the technology also proved relatively simple, reliable and secure.

Considering VPNs foolproof, however, leads to a false sense of security. Following state-sponsored attacks that used compromised VPNs to enable exploitative attacks, organizations received a wakeup call that VPN accounts require close monitoring and safeguarding too.

With proper security practices, VPNs continue to effectively fulfill an essential need reliably and securely connecting remote employees, branch offices, authorized partners and other systems. Yet VPN connection errors continue to inevitably arise.

Often, Windows server-powered VPN connection issues that arise often fall into one of four categories:

The VPN connection is rejected.
An unauthorized connection is accepted.
Locations beyond the VPN server prove unreachable.
A tunnel cannot be established.

Here’s how to resolve these common Windows Server-powered VPN connection errors.

SEE: No VPN? Why your company needs one and how to pick the best provider (TechRepublic)

Working with the Windows Server Routing and Remote Access console

Once a VPN is set up using a Windows Server, connection issues occasionally occur, even when a connection previously worked properly. Troubleshooting often involves working with Windows servers’ Routing and Remote Access console snap-in tool, which is where Microsoft concentrates many VPN configuration settings.

The Routing and Remote Access snap-in lives within the Microsoft Management Console, known as the MMC. There are multiple ways to access the MMC. You can select the console from the Start menu’s Programs options, within the Administrative Tools folder within Windows server’s Control Panel or by typing mmc at a command prompt. You can also reach the MMC by pressing the Windows key and the letter R simultaneously and entering mmc and pressing the Enter key.

While the actual user interface and menu options occasionally change subtly between specific server versions, administrators should be able to navigate the various consoles — whether working with an older version or the current Windows Server 2022 iteration — using the same approach.

How to fix the four biggest problems with failed VPN connections

1: The VPN connection is rejected.

Having a VPN client’s connection rejected is perhaps the most common VPN problem. Part of the reason this problem is so common is that many issues can cause a connection to be rejected.

If the Windows server-powered VPN is rejecting client connections, the first thing you need to do is confirm the Routing and Remote Access Service is actually running on the Windows server. You can check by opening the Windows server’s Services console, which you can access by clicking Start | Control Panel | Administrative Tools | Services. With the Services console open, navigate within the list of services to the Routing and Remote Access entry ensure its service is running.

As TechRepublic’s Brandon Vigliarolo demonstrates within his video at the start of this article, the Services console displays the status of the Routing and Remote Access entry. From within the Services console and with the Routing and Remote Access entry highlighted, you can click Start the Service or right-click the entry and select Restart. If the RRAS service was set to Manual or Disabled, you can open the entry, change the Startup Type to Automatic and then click Start and OK.

After confirming the RRAS service is running, and as Vigliarolo also reviews, it’s a good idea to test the connection by pinging the VPN server first by IP address, then by its fully qualified domain name. If you encounter errors, it’s likely a DNS problem is occurring and you can turn your attention to resolving that issue.

If the VPN server pings work, though, and you’re still having connection issues, turn your attention to addressing a potential authentication mismatch. Sometimes the VPN client and VPN server are set to using different authentication methods.

Confirm whether an authentication error is the problem by opening the server console. Yet another method of accessing the MMC is to type Control+R to open a command prompt in which you can type mmc and hit Enter or click OK.

With the console open, navigate to the Routing and Remote Access entry. If the entry isn’t present, click File, select Add/Remove Snap-in, choose the Routing and Remote Access option from the choices and click Add, then OK.

With the Routing and Remote Access snap-in added, right-click on the VPN server and click Properties. Then, review the Security tab to confirm the authentication method. Windows Authentication is the most common, although a different option such as RADIUS may be in place. Ensure the VPN client is set to the authentication method specified within the Security tab.

SEE: Check these settings in Windows Server to fix VPN errors (TechRepublic)

More things to check

Typically the items just reviewed are responsible for most VPN connection refusal errors. But other fundamentals must be correct, too.

For example, if the Windows Server hosting the VPN hasn’t joined the Windows domain, the server will be unable to authenticate logins. You’ll first have to connect the server to the domain.

IP addresses are another fundamental element for which administration must be properly set. Each Web-based VPN connection usually uses two different IP addresses for the VPN client computer. The first IP address is the one that was assigned by the client’s ISP. This is the IP address that’s used to establish the initial TCP/IP connection to the VPN server over the Internet. However, once the client attaches to the VPN server, the VPN server assigns the client a secondary IP address. This IP address typically possesses the same subnet as the local network and thus allows the client to communicate with the local network.

When you set up the VPN server, you must configure a DHCP server to assign addresses to clients, or you can create a bank of IP addresses to assign to clients directly from the VPN server. In either case, if the server runs out of valid IP addresses, it will be unable to assign an address to the client and the connection will be refused.

For DHCP server environments, a common setup error is specifying an incorrect NIC. If you right-click on the VPN server within the Routing and Remote Access snap-in and select the Properties command from the resulting shortcut menu, you should see the server’s properties. The corresponding IP tab contains settings that permit specifying the DHCP source. Ensure that if the DHCP server option is enabled, the appropriate network adapter is selected. You must select a network adapter that has a TCP/IP path to the DHCP server.

2: An unauthorized connection is accepted.

Next, let’s review the opposite problem, in which unauthorized connections are accepted. This problem is much less common than not connecting, but the problem is much more serious because of the potential security issues and resultant unauthorized traffic.

If you look at a user’s properties sheet in the Active Directory Users and Computers console, the Dial In tab usually contains an option to control access through the remote access policy. If this option is selected and the effective remote access policy is set to allow remote access, the user will be able to attach to the VPN.

Although I have been unable to re-create the situation personally, I have heard rumors that a bug exists in older Windows servers that can cause the connection to be accepted even if the effective remote access policy is set to deny a user’s connection. Therefore, and especially on older server platforms, it’s best to allow or deny connections directly through the Active Directory Users and Computers console.

A host of other security fundamentals should be in place, too, to help prevent unauthorized VPN access. Unnecessary VPN accounts should always be disabled and even deleted, when possible. Users should be required to change their corresponding passwords frequently, and those passwords should need to meet complexity requirements.

Multi-factor authentication should be required for all VPN connections, and network firewalls and security services should continually monitor for unauthorized or suspicious connections to generate high-priority alerts whenever possible issues surface. Implementing those steps will help reduce the likelihood an unauthorized connection is accepted.

3: Locations beyond the VPN server prove unreachable.

Another common VPN problem is that a connection is successfully established but the remote user is unable to access the network beyond the VPN server. By far, the most common cause of this problem is that permission hasn’t been granted for the user to access the entire network.

To allow a user to access the entire network, go to the Routing and Remote Access console and right-click on the VPN server that’s having the problem. Select the Properties command from the resulting shortcut menu to display the server’s properties sheet, then select the properties sheet’s IP tab. At the top of the IP tab is an Enable IP Routing check box. If this check box is enabled, VPN users will be able to access the rest of the network, assuming network firewalls and security-as-a-service settings permit. If the checkbox is not selected, these users will be able to access only the VPN server, but nothing beyond.

The problem could also be related to other routing issues. For example, if a user is dialing directly into the VPN server, it’s usually best to configure a static route between the client and the server.

You can configure a static route by going to the Dial In tab of the user’s properties sheet in Active Directory Users and Computers and selecting the Apply A Static Route check box. This will cause Windows to display the Static Routes dialog box. Click the Add Route button and then enter the destination IP address and network mask in the space provided. The metric should be left at 1.

If you’re using a DHCP server to assign IP addresses to clients, there are a couple of other problems that could cause users not to be able to go beyond the VPN server. One such problem is that of duplicate IP addresses. If the DHCP server assigns the user an IP address that is already in use elsewhere on the network, Windows will detect the conflict and prevent the user from accessing the rest of the network.

Another common problem is the user not receiving an address at all. Most of the time, if the DHCP server can’t assign the user an IP address, the connection won’t make it this far. However, there are situations in which an address assignment fails, so Windows automatically assigns the user an address from the 169.254.x.x range. If the client is assigned an address in a range that’s not present within the system’s routing tables, the user will be unable to navigate the network beyond the VPN server.

Other issues can contribute to this problem, too. Ensure the resources the user is attempting to access are actually on the network to which the user is connecting.

With the growing number of servers, cloud platforms and application as a service options, it’s possible the user is seeking a resource on the wrong network or on a subnet to which the network the user connected can’t reach. A VPN connection to the other subnet might, in fact, be required. A firewall or security as a service solution could also be to blame, so don’t forget to review those solutions’ settings, if such components are present between the VPN server and the resources the user seeks to reach.

4: A tunnel cannot be established.

If everything seems to be working well, but you can’t seem to establish a tunnel between the client and the server, there are two main possibilities of what could be causing the problem.

The first possibility is that one or more of the routers involved is performing IP packet filtering. IP packet filtering could prevent IP tunnel traffic. I recommend checking the client, the server and any machines in between for IP packet filters. You can do this by clicking the Advanced button on each machine’s TCP/IP Properties sheet, selecting the Options tab from the Advanced TCP/IP Settings Properties sheet, selecting TCP/IP Filtering and clicking the Properties button.

The other possibility is that a proxy server is standing between the client and the VPN server. A proxy server performs NAT translation on all traffic flowing between the client and the Internet. This means that packets appear to be coming from the proxy server rather than from the client itself. In some cases, this interaction could prevent a tunnel from being established, especially if the VPN server is expecting the client to have a specific IP address.

You must also keep in mind that older or low-end proxy servers (or NAT firewalls) don’t support the L2TP, IPSec or PPTP protocols that are often used for VPN connections.

In other cases, firewall security services or security as a service solutions might be blocking the formation of a VPN tunnel. Review the settings within those various devices or services to ensure the Windows server-powered VPN traffic is properly supported.