802.1X Remote desktop (RDP) user authentication

When connecting through remote desktop (RDP) to machines that are connected to 802.1X enabled switch ports (for example in Cisco ISE installations), connectivity issues, and session failures and disconnection issues could happen. Most of these issues are related to the fact that native 802.1X supplicants don't authenticate and authorize users that connect through RDP. Access lists on port are left as you have machine with no user logged on it. In most cases these ACLs allow only traffic for logging user on domain etc., and remote desktop traffic is not allowed by these ACLs. We have resolved this issue by using Cisco AnyConnect Secure Mobility Client with Network Access Manager - NAM module, as it behaves better and eliminates this problem by authenticating and authorizing user that is connecting through RDP and granting all rights and expected ACLs.

Cisco ISE - duplicate IP address on Windows 7

Windows 7 sometimes reports IP address conflict (duplicate IP address warning message) when connected to 802.1X enabled ports while device tracking feature is used on switch. Device tracking is used in most Cisco ISE designs. You can check if you have device tracking on switch with:

switch#sh run | inc trackin
ip device tracking
switch#

This usually happens on Windows Vista, Windows 2008 and Windows 7 clients, when disconnecting and connecting again (or shut no shut on switch), after locking and unlocking, or logging off and then logging on. Reason behind this is that windows machines could send ARP packets at the same time the switch is probing for device status. Possible solutions are:
Solution 1 - Delay ip device tracking ARP probe for few seconds (you should have IOS version that supports this command).

Switch(config)# ip device tracking [probe {delay interval}]

ARP packet will be sent few seconds after link-up, and it will not happen at same time when windows 7 clients send first ARP packets. For example:

switch(config)#ip device tracking probe delay 5

Solution 2 - Disable gratuitous ARP in Windows registry [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters] Set ArpRetryCount (32-bit DWORD) to 0 If you decide to use second method, please be sure to manage duplicate address problem with some other controls on switches or elsewhere.

With device tracking probe delay we had really good results, and believe there is no need for registry tuning (and problem with it is that when new machine with default registry settings is connected, you will again get same duplicate IP address problem).

GETVPN packets almost silently dropped when onboard VPN hardware is used

show crypto ipsec sa shows no (counter is zero all the time) packets are encapsulated or decapsulated. This is fine, but strange part is that send and receive error counters are also 0 all the time.

Group_member#show crypto ipsec sa
interface: GigabitEthernet0/0
    Crypto map tag: getvpn-map, local addr 10.10.10.10

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 0.0.0.0 port 848
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
    #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 10.10.10.10, remote crypto endpt.: 0.0.0.0
     path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0
     current outbound spi: 0x393A7B96(960134038)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0x393A7B96(960134038)
        transform: esp-256-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2045, flow_id: Onboard VPN:45, sibling_flags 80000040, crypto map: getvpn-map
        sa timing: remaining key lifetime (sec): (4219)
        IV size: 16 bytes
        replay detection support: Y  replay window size: 5
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0x393A7B96(960134038)
        transform: esp-256-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2046, flow_id: Onboard VPN:46, sibling_flags 80000040, crypto map: getvpn-map
        sa timing: remaining key lifetime (sec): (4219)
        IV size: 16 bytes
        replay detection support: Y  replay window size: 5
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

CEF is turned off, so debug ip packet is able to show needed info.

Group_member#(config-if)#no ip route-cache   

Packet debuging is turned on for problematic traffic flow, and also few other debugs.

Group_member#debug ip packet 100
IP packet debugging is on for access list 100
Group_member#debug crypto socket
Crypto secure socket events debugging is on
Group_member#debug cry ipsec error
Crypto IPSEC Error debugging is on

IP: s=10.10.1.1 (GigabitEthernet0/1), d=10.20.1.1, len 100, input feature, MCI Check(63), rtype 0, forus FALSE, sendself FALSE, mtu 0
IP: s=10.10.1.1 (GigabitEthernet0/1), d=10.20.1.1 (GigabitEthernet0/0), len 100, output feature, IPSec output classification(25), rtype 1, forus FALSE, sendself FALSE, mtu 0
Before encryption:
3F405990:                   45000064 7A760000          E..dzv..
3F4059A0: FE01ABC6 0A0A4101 0A504101 0800FFD4  ~.+F..A..PA....T
3F4059B0: 026A0000 00000008 B93CC2C6 ABCDABCD  .j......9BBF+M+M
3F4059C0: ABCDABCD ABCDABCD ABCDABCD ABCDABCD  +M+M+M+M+M+M+M+M
3F4059D0: ABCDABCD ABCDABCD ABCDABCD ABCDABCD  +M+M+M+M+M+M+M+M
3F4059E0: ABCDABCD ABCDABCD ABCDABCD ABCDABCD  +M+M+M+M+M+M+M+M
3F4059F0: ABCDABCD ABCDABCD ABCDABCD           +M+M+M+M+M+M 
pak 48564F80 consumed in output feature , packet consumed, IPSec: to crypto engine(54), rtype 1, forus FALSE, sendself FALSE, mtu 0

Everything looks as it should.

Show crypto tech-support finally gives a trace to follow in its show cry engine accelerator statistic part, pkts dropped error counter and ppq full error counter:

Group_member#sh crypto tech-support
...
------------------ show cry engine accelerator statistic ------------------



Device:   Onboard VPN
Location: Onboard: 0
        :Statistics for encryption device since the last clear
         of counters 86585 seconds ago
                      0 packets in                         128 packets out        
                      0 bytes in                          9840 bytes out          
                      0 paks/sec in                          0 paks/sec out       
                      0 Kbits/sec in                         0 Kbits/sec out      
                      0 packets decrypted                    0 packets encrypted  
                      0 bytes before decrypt              9840 bytes encrypted    
                      0 bytes decrypted                      0 bytes after encrypt
                      0 packets decompressed                 0 packets compressed 
                      0 bytes before decomp                  0 bytes before comp  
                      0 bytes after decomp                   0 bytes after comp   
                      0 packets bypass decompr               0 packets bypass compres
                      0 bytes bypass decompres               0 bytes bypass compressi
                      0 packets not decompress               0 packets not compressed
                      0 bytes not decompressed               0 bytes not compressed
                  1.0:1 compression ratio                1.0:1 overall
                Last 5 minutes:
                      0 packets in                           0 packets out        
                      0 paks/sec in                          0 paks/sec out       
                      0 bits/sec in                          0 bits/sec out       
                      0 bytes decrypted                      0 bytes encrypted    
                      0 Kbits/sec decrypted                  0 Kbits/sec encrypted
                  1.0:1 compression ratio                1.0:1 overall

 Errors:
                    168 pkts dropped                       168 ppq full           
                      0 tx parts overflow                    0 rx parts overflow  
                      0 replenishment failure                0 zero len           
                      0 flow inputs bad                      0 cmd invalid        
                      0 IPV4 len                             0 IPV6 len           
                      0 algor invalid      
                      0 bad shadow particle                  0 algor disabled     
                      0 pre tx fail                          0 dma error          
                      0 dbit miss                            0 pipeline abort     
                      0 failsafe timeout                     0 reserv             
                      0 bad sz count                         0 bad shdw           
                      0 bad flow tx                          0 spi mismatch       
                      0 bad flow rx                          0 auth fail          
                      0 udm fs fail                          0 pad fail           
                      0 addr limit fixup fail                0 seq fail           
                      0 quad fix sp                          0 quad fix mp        
                      0 quad fix cont 
...                   

It seems that there is a problem when GETVPN is used with onboard crypto engine.
To confirm this assumption turn off accelerator (no crypto engine accelerator):

Group_member(config)#no cry eng accelerator
...switching to SW crypto engine

And the problem is gone and packets are processed as they should.

It turns out that there is a problem with some 1800, 2800 and 3800 routers with onboard crypto when some 12.4T IOS versions are used. "ppq full" counter going up in show crypto engine accelerator statistic identifies this issue. Upgrade to a version that resolves this issue is a complete solution, and no crypto engine accelerator a temporary workaround.

To check whether onboard hardware VPN Module accelerator or software VPN implementation is used you can use show crypto engine config:

Group_member#sh cry eng config


        crypto engine name:  Virtual Private Network (VPN) Module
        crypto engine type:  hardware
                     State:  Enabled
                  Location:  onboard 0
              Product Name:  Onboard-VPN
                FW Version:  01100200
              Time running:  491 seconds
               Compression:  Yes
                       DES:  Yes
                     3 DES:  Yes
                   AES CBC:  Yes (128,192,256)
                  AES CNTR:  No
     Maximum buffer length:  4096
          Maximum DH index:  0300
          Maximum SA index:  0300
        Maximum Flow index:  0600
      Maximum RSA key size:  2048

        crypto lib version:  20.0.0

     crypto engine in slot:  0
                  platform:  VPN hardware accelerator
        crypto lib version:  20.0.0
     
Group_member(config)#no cry eng accelerator

Group_member(config-if)#do sh cry eng config

        crypto engine name:  Cisco VPN Software Implementation
        crypto engine type:  software
             serial number:  FJ34208J
       crypto engine state:  installed
     crypto engine in slot:  N/A
                  platform:  Cisco Software Crypto Engine
        crypto lib version:  20.0.0

Short problem description:
GETVPN crypto engine accelerator ppq full errors
Packets are almost silently dropped while using hardware crypto engine. When crypto processing is switched to software (no cry eng accelerator), everything works fine.
Only place where we have seen trace of this error is in show crypto engine accelerator statistic command in "ppq full" statistics that are increasing by 1 for every packet that should be encrypted.