Last updated at Fri, 14 Jun 2024 20:26:34 GMT

On Monday, October 16, Cisco’s Talos group published a blog on an active threat campaign exploiting CVE-2023-20198, a “previously unknown” zero-day vulnerability in the web UI component of Cisco IOS XE software. IOS XE is an operating system that runs on a wide range of Cisco networking devices, including routers, switches, wireless controllers, access points, and more. Successful exploitation of CVE-2023-20198 allows a remote, unauthenticated attacker to create an account on an affected device and use that account to obtain full administrator privileges, effectively enabling a complete takeover of the system.

There was no patch for CVE-2023-20198 at time of disclosure (October 17, 2023). Cisco has released fixed versions for a range of solutions as of October 22. As Cisco Talos noted in their blog, the vulnerability has been exploited in the wild, and there appeared to be a significant number of devices running IOS XE on the public internet as of October 17. Estimates of internet-exposed devices running IOS XE vary, but the attack surface area does appear to be relatively large; one estimate puts the exposed device population at 140K+.

On October 20, Cisco updated their advisory on CVE-2023-20198 to reflect that the attack chain their team observed actually included two zero-day vulnerabilities, not just the one:

"The attacker first exploited CVE-2023-20198 to gain initial access and issued a privilege 15 command to create a local user and password combination. This allowed the user to log in with normal user access.

The attacker then exploited another component of the web UI feature, leveraging the new local user to elevate privilege to root and write the implant to the file system. Cisco has assigned CVE-2023-20273 to this issue.

CVE-2023-20198 has been assigned a CVSS Score of 10.0.
CVE-2023-20273 has been assigned a CVSS Score of 7.2.

Both of these CVEs are being tracked by CSCwh87343."

Additional activity has included deployment of an implant that allows the attacker to execute arbitrary commands at the system level or IOS level. Cisco has an extensive description of the malicious behavior they’ve observed here.

Affected products

Cisco’s public advisory on CVE-2023-20198 and CVE-2023-20273 says that Cisco IOS XE software is vulnerable if the web UI feature is enabled (the UI is enabled through the ip http server or ip http secure-server commands). Cisco does not offer a list of products that definitively run IOS XE, but their product page for IOS XE lists some, including the Catalyst, ASR, and NCS families.

According to the advisory, customers can determine whether the HTTP Server feature is enabled for a system, by logging into the system and using the show running-config | include ip http server|secure|active command in the CLI to check for the presence of the ip http server command or the ip http secure-server command in the global configuration. The presence of either command or both commands in the system configuration indicates that the web UI feature is enabled (and that the system is therefore vulnerable).

Cisco’s advisory also specifies that if the ip http server command is present and the configuration also contains ip http active-session-modules none, the vulnerability is not exploitable over HTTP. If the ip http secure-server command is present and the configuration also contains ip http secure-active-session-modules none, the vulnerability is not exploitable over HTTPS.

Mitigation guidance

As of October 22, Cisco has released fixed versions of IOS XE that remediate CVE-2023-20198 for a range of platforms across their solution portfolio (e.g., SDWAN, various routers and switches). Organizations should disable the web UI (HTTP Server) component on internet-facing systems on an emergency basis before applying patches. Organizations should also reboot their devices.

To disable the HTTP Server feature, use the no ip http server or no ip http secure-server command in global configuration mode. Per Cisco’s advisory, if both the HTTP server and HTTPS server are in use, both commands are required to disable the HTTP Server feature. Organizations should also avoid exposing the web UI and management services to the internet or to untrusted networks.

Disabling the web UI component of IOS XE systems and limiting internet exposure reduces risk from known attack vectors, but notably does not mitigate risk from implants that may have already been successfully deployed on vulnerable systems. Rapid7 recommends invoking incident response procedures where possible to prioritize hunting for indicators of compromise Cisco has shared, listed below.

Cisco-observed attacker behavior

The Cisco Talos blog on CVE-2023-21098 has a full analysis of the implant they’ve observed being deployed as part of this threat campaign. We strongly recommend reading the analysis in its entirety. The implant is saved under the file path /usr/binos/conf/nginx-conf/cisco_service.conf that contains two variable strings made up of hexadecimal characters. While the implant is not persistent (a device reboot will remove it), the attacker-created local user accounts are.

Cisco observed the threat actor exploiting CVE-2021-1435, which was patched in 2021, to install the implant after gaining access to a device vulnerable to CVE-2023-20198. Talos also notes that they have seen devices fully patched against CVE-2021-1435 getting the implant successfully installed “through an as of yet undetermined mechanism.”

Rapid7-observed attacker behavior

Rapid7 MDR has so far identified a small number of instances where CVE-2023-20198 was exploited in customer environments, including multiple instances of exploitation within the same customer environment on the same day. The indicators of compromise our team has identified with available evidence indicate the use of techniques similar to those described by Cisco Talos.

Rapid7 identified variations of techniques in the course of our investigations. The first malicious activity performed on the system post-exploitation was associated with the admin account. The following is an excerpt from this log file:
%SYS-5-CONFIG_P: Configured programmatically by process SEP_webui_wsma_http from console as admin on vty1
The threat actor created the local account cisco_support using the command username cisco_support privilege 15 algorithm-type sha256 secret * under user context admin. The threat actor then authenticated to the system using this newly created cisco_support account and began running several commands, including the following:

show running-config
show voice register global
show dial-peer voice summary
show platform
show flow monitor
show platform
show platform software iox-service
show iox-service
dir bootflash:
dir flash:
clear logging
no username cisco_support
no username cisco_tac_admin
no username cisco_sys_manager

Upon completion of these commands, the threat actor deleted the account cisco_support. The accounts cisco_tac_admin and cisco_sys_manager were also deleted, but Rapid7 did not observe account creation commands associated with these accounts within available logs.

The threat actor also executed the clear logging command to clear system logging and cover their tracks. Rapid7 identified logging for the second exploitation on October 12, 2023, but could not review logs for the first intrusion because the logs had been cleared.

Evidence indicated that the last action performed by the threat actor relates to a file named aaa:
%WEBUI-6-INSTALL_OPERATION_INFO: User: cisco_support, Install Operation: ADD aaa

When comparing the two intrusions that occurred within the same environment on October 12, there are slight differences in observed techniques. For example, log clearing was only performed within the first exploitation, while the second exploitation included additional directory viewing commands.

Indicators of compromise

The Cisco Talos blog on CVE-2023-20198 directs organizations to look for unexplained or newly created users on devices running IOS XE. One way of identifying whether the implant observed by Talos is present is to run the following command against the device, where the "DEVICEIP” portion is a placeholder for the IP address of the device to check:

curl -k -X POST "http[:]//DEVICEIP/webui/logoutconfirm.html?logon_hash=1"

The command above will execute a request to the device’s Web UI to see if the implant is present. If the request returns a hexadecimal string, the implant is present (note that the web server must have been restarted by the attacker after the implant was deployed for the implant to have become active). Per Cisco’s blog, the above check should use the HTTP scheme if the device is only configured for an insecure web interface.

Additional Cisco IOCs

  • 5.149.249[.]74
  • 154.53.56[.]231

Usernames:

  • cisco_tac_admin
  • cisco_support

Cisco Talos also advises performing the following checks to determine whether a device may have been compromised:

Check the system logs for the presence of any of the following log messages where “user” could be cisco_tac_admin, cisco_support or any configured, local user that is unknown to the network administrator:

  • %SYS-5-CONFIG_P: Configured programmatically by process SEP_webui_wsma_http from console as user on line

  • %SEC_LOGIN-5-WEBLOGIN_SUCCESS: Login Success [user: user] [Source: source_IP_address] at 03:42:13 UTC Wed Oct 11 2023

Note: The %SYS-5-CONFIG_P message will be present for each instance that a user has accessed the web UI. The indicator to look for is new or unknown usernames present in the message.

Organizations should also check the system logs for the following message where filename is an unknown filename that does not correlate with an expected file installation action:

  • %WEBUI-6-INSTALL_OPERATION_INFO: User: username, Install Operation: ADD filename

Rapid7 customers

As of October 17, InsightVM and Nexpose customers can assess their exposure to CVE-2023-20198 with an authenticated vulnerability check that looks for Cisco IOS XE devices with the web UI enabled. We expect to release an update to this check on October 24 to reflect fixed version availability.

InsightIDR and Rapid7 MDR customers have existing detection coverage through Rapid7's expansive library of detection rules. The following detection rules are deployed and alerting on activity related to this vulnerability via the IP addresses provided by Cisco:

  • Network Flow - CURRENT_EVENTS Related IP Observed
  • Suspicious Connection - CURRENT_EVENTS Related IP Observed

Updates

October 17, 2023: Updated with Rapid7-observed attacker behavior and IOCs.

October 23, 2023: Updated to reflect the disclosure of a second zero-day vulnerability, CVE-2023-20273. Also updated to note that Cisco has released a patch for CVE-2023-20198 across a number of affected platforms. Rapid7 expects to release an update to the vulnerability check for CVE-2023-20198 on October 24 to detect patched versions of IOS XE.