Cisco is honored to be a associate of the Black Hat NOC (Community Operations Middle), and this was our seventh yr supporting Black Hat Asia. Cisco is the Official Cellular Machine Administration, Malware Evaluation and DNS (Area Title Service) Supplier.
We work with different official suppliers to carry the {hardware}, software program and engineers to construct and safe the community, for our joint buyer: Black Hat.
- Arista: Community Gear
- Corelight: Community Analytics and Detection
- MyRepublic: Broadband
- NetWitness: Menace Detection & Response, Id
- Palo Alto Networks: Community Safety Platform
The first mission within the NOC is community resilience. The companions additionally present built-in safety, visibility and automation, a SOC (Safety Operations Middle) contained in the NOC.
On screens outdoors the NOC have been displayed associate dashboards for the attendees to view the quantity and safety of the community site visitors.
It All Began with Malware
Cisco joined the Black Hat NOC in 2016, when requested to offer automated malware evaluation with Thread Grid. The Cisco contributions to the community and safety operations advanced, with the wants of the shopper, to incorporate extra elements of the Cisco Safety Cloud.
The NOC leaders allowed Cisco (and the opposite NOC companions) to usher in further software program to make our inside work extra environment friendly and have better visibility; nonetheless, Cisco will not be the official supplier for Prolonged Detection & Response, Community Detection & Response or Collaboration.
- Breach Safety Suite
- Cisco XDR: Menace Looking / Menace Intelligence Enrichment / Government dashboards / Automation with Webex
- Cisco XDR Analytics (Previously Safe Cloud Analytics / Stealthwatch Cloud): community site visitors visibility and risk detection
- Cisco Webex: Incident notification and staff collaboration
The Cisco XDR Command Middle dashboard tiles made it simple to see the standing of every of the related Cisco Safety applied sciences, and the standing of ThousandEyes brokers.
When the companions deploy to every convention, we arrange a world class community and safety operations middle in three days. Our aim stays community up time and creating higher built-in visibility and automation. Black Hat has the choose of the safety business instruments and no firm can sponsor/purchase their manner into the NOC. It’s invitation solely, with the intention of range in companions, and an expectation of full collaboration.
As a NOC staff comprised of many applied sciences and firms, we’re repeatedly innovating and integrating, to offer an total SOC cybersecurity structure answer. We sit up for persevering with the work with associate Palo Alto Networks, for additional automation at Black Hat USA 2024.
Under are the Cisco XDR integrations for Black Hat Asia, empowering analysts to research Indicators of Compromise (IOC) in a short time, with one search.
We recognize alphaMountain.ai, Pulsedive and Recorded Future donating full licenses to Cisco, to be used within the Black Hat Asia 2024 NOC.
An instance of that is an investigation of a doubtlessly malicious exercise on the twond day of Coaching. An IP handle was recognized by NetWitness for potential geolocation leakage.
Investigation of the IP correlated the syslog sightings from the associate applied sciences within the NetWitness logs, with risk intelligence from Pulsedive, Recorded Future, alphaMountain and others.
Reviewing the DNS logs and the small print of the packet seize in each Corelight and NetWitness, it was confirmed no geolocation knowledge was leaked and it was a part of a Coaching course. The exercise would have been blocked in a manufacturing atmosphere.
A core built-in workflow within the Black Hat NOC is NetWitness and Corelight sending suspicious recordsdata to Safe Malware Analytics. Over 4,900 samples have been submitted.
The NOC analysts additionally used Malware Analytics to research suspicious domains, with out the chance of an infection. Reasonably than going to the web site on a company or Black Hat property, we have been capable of work together with the web site within the glovebox, together with downloading and putting in the web site payload.
Detonating recordsdata or searching web sites in Safe Malware Analytics protects the analysts from unintentional an infection.
We noticed a collection of comparable (however totally different hash values) exploit kits downloaded on the primary day within the Enterprise Corridor. The downloads have been on the convention Wi-Fi and never in a Coaching course, so the occasion needed to be investigated to substantiate there was not an assault on the attendees. Working with the Corelight staff, the NOC responders parsed the site visitors and confirmed it was a Seize the Flag occasion, which continued into the final day of the convention.
Menace Hunters’ Story, by Aditya Raghavan and Shaun Coulter
Within the Black Hat Asia 2024 NOC, Shaun staffed the morning shifts, and Aditya the afternoon shifts, as risk hunters centered on the Cisco XDR and Safe Malware Analytics consoles. Mornings have been often fairly chill. Nevertheless, and for some heretofore unknown (espresso associated?) cause, the exercise ramped up within the afternoon on most days, main Aditya to a spot of “concerned pleasure”, and Shaun to a spot of tormented jealousy :D. With dogged willpower each hunters spent their time reviewing alerts, actions, and performed IOC scans utilizing XDR Examine. They reviewed submitted samples and community logs for indicators of intrusion or suspicious exercise.
Utilizing Safe Malware analytics, they dissected malware samples, analyzed phishing campaigns, and scrutinized community site visitors patterns for anomalies. Quite a few alerts flagged as spikes in site visitors from sudden sources, unusual locations and odd variants of malicious code popped up a number of instances, initiating thorough investigations. Typically, they traced the anomaly to a licensed Black Hat Coaching or Briefing supply and closed such instances as “Black Hat Optimistic”; that means you wouldn’t enable this in your manufacturing community, however for Black Hat, it’s enterprise as common. Since Black Hat is a convention designed for studying about offensive safety, these malware samples are anticipated, and marked as such.
Fortunately or unthankfully, because the system tuning was accomplished, most alerts raised have been as above and anticipated or really ‘close to misses’ – objects that warrant investigation however didn’t lengthen to impactful behaviours, as we have been capable of cease them in time.
On the primary day of Briefings, as Shaun is dutifully poring via the console of Safe Malware Analytics, in walks Aditya to alleviate the shift. Greetings apart, Shaun shortly pivots over excitedly “Brother, I need to present you a few fascinating issues.” Aditya’s curiosity is piqued, and Shaun opens a brand new dashboard exhibiting one of many just lately launched options of Cisco XDR – MITRE ATT&CK ® Protection Map.
This new functionality shortly shows all of the ways and strategies within the MITRE ATT&CK® matrix for which Cisco XDR has detections/protection. Along with the XDR Native, detections from Safe Endpoint and Safe Malware Analytics are additionally used to derive the protection map making it a holistic view. This view permits the person to visualise the detections of XDR natively, in addition to the built-in options and establish the scope of protection and importantly, map out the gaps for future consideration. Because of the Cisco Talos staff, all options inside the Cisco Breach Safety Suite are mapped in the present day and this could be rolled out to incorporate different suites and options, together with 3rd occasion integrations, quickly.
As our risk hunters geek out on the behind-the-scenes stuff on XDR, Jessica politely calls out “Adi. Shaun. Guys, there may be some new exercise on Umbrella. Are you able to look into it?” Nudged again to actuality, our risk hunters get to work – discovering needles within the stack of needles at Black Hat because it was rightly put by Grifter! Speaking about that, the brand new exercise seems to be a question for a website labeled as a Command & Management (C&C) area. Let’s dig into it.
A fast look into Umbrella Exercise Search exhibits the most recent site visitors exercise matching the C&C class that was allowed. Increasing the small print pane, we will see the area being queried and the identification of the endpoint issuing the question which seems to be from the ‘Hacking Enterprises 2024 Pink Staff’. That may be a respectable Coaching class at Black Hat Asia 2024. We pivot over to Umbrella Examine and see the rationale for this area being categorized as C&C and its indicators.
Let’s head over to XDR and question this observable in opposition to all of the built-in options for extra intel. We shortly get a visible related graph and tabulated occasions on all of the related intel. The mixing with NetWitness Logs supplies us with occasions associated to that area, in addition to populating the graph with these relationships, together with the Umbrella occasion which was the supply for this hunt.
Trying on the proof, this turned out to be one other needle! Nothing untoward right here, we labeled this as a ‘Black Hat Optimistic’ and moved on. Because the afternoon shift winds down, the staff is discussing potential locations for dinner and there may be at all times dessert to sit up for on the finish. Aditya and Ryan have been pining for wealthy ice cream and House Greatest Dessert seems to be the best answer for the ask. Within the NOC, the best answer is sort of at all times teamwork with all our companions.
One such occasion was when a Corelight hunter picked up a spike of site visitors to uncommon locations. These seem like DNS queries to a bunch of C&C domains. We shortly delve into Umbrella exhibiting us all of the domains being queried in a brief window and most of them being Malware and/or C&C categorized. This seems to be a system both being compromised or somebody deliberately doing a take a look at / recon for these domains.
Let’s examine a few of these domains in XDR. We will see a variety of crimson icons on this visualization! The truth is, each queried area is assessed as Malicious and recognized to host different malicious content material. This doesn’t look anticipated for certain and that places the intentional take a look at / recon idea to relaxation shortly. Ben Reardon, the hunter from Corelight, places it succinctly “This field is pwned six methods to Sunday!” What else can we discover about this technique then?
Trying on the DHCP logs for the IP handle, the Corelight hunter was capable of pinpoint the gadget MAC handle and hostname, which resembled a reputation. A brief Google search later, now we have a possible gadget proprietor and the truth that he was delivering a session at Black Hat in one of many rooms subsequent door! A brief dialog with the particular person after his session ensued, the place the NOC leads suggested the NOC’s findings on his compromised system. He was grateful for the discovering and reached out for extra context. This one turned out to be a ‘True Optimistic.’
The next day, the staff has zeroed in on Turkish meals for the night. Ryan halts Shaun as he departs on the finish of his shift and calls for his lodge identify and room quantity. “I’m gonna come knock at your door and wake you up tonight, man. I imply it. No day is simply too lengthy. I used to do my shifts on three hours of sleep. Now, let’s go!” Ryan is deadpan critical. That’s what we thought whereas investigating our subsequent potential malware discovering.
One other occasion on the Umbrella console involves our consideration and this time it’s a question for a website categorized as Malware. The supply endpoint is shortly recognized from the Id and Umbrella examine tells us this area is a part of the Malware block record. In a traditional manufacturing community, this could ideally be blocked.
Black Hat isn’t your regular manufacturing community, and it attracts all types of safety folks. And that’s precisely what it turned out to be this time. The Nationwide College of Singapore has a gaggle organizing common seize the flag (CTF) occasions and is working an analogous get-together at Black Hat. Go NUS Greyhats!
Actions involving malware what can be blocked on a company community have to be allowed, inside the confines of Black Hat Code of Conduct.
Community Observability with ThousandEyes, by Adam Kilgore and Patrick Yong
Deploying ThousandEyes at Black Hat is a rigorous course of involving a variety of {hardware} (some proven under), configuration, testing, troubleshooting, and working across the convention middle.
Along with our typical deployment duties, we carried out a number of enhancements to the service. These enhancements included an overhaul of the dashboards to indicate granular knowledge for every convention room, alongside combination knowledge for the whole convention; and higher labeling and group of deployed brokers.
The ThousandEyes dashboard was projected on the massive display screen within the NOC, for alerting on any community points, previous to stories from customers.
On the troubleshooting facet, we improved our log evaluation and assortment strategies and arrange centralized monitoring of wi-fi knowledge. These efforts contributed to enhancements in visibility and agent uptime all through the convention.
Throughout the preliminary two days of Coaching classes at Black Hat, ThousandEyes brokers confirmed solely minor deviations from baseline because the Coaching classes got here on-line. Because the Coaching classes continued, efficiency was secure, with solely uncommon alerts for minor degraded throughput or average latency spikes. On Thursday, all of the two-day Coaching classes have been closed, and the convention shifted in direction of Briefings, alongside two four-day Coaching classes that ran for the convention’s size. With begin of Briefings and opening the Enterprise Corridor, headcounts drastically elevated. ThousandEyes noticed degraded efficiency on the community, primarily within the giant convention rooms internet hosting the Briefings. The under picture exhibits a take a look at end result from the Hibiscus 3610 ballroom:
The community path above exhibits heavy latency on the primary hyperlink to the default gateway, compounded by one other excessive latency hyperlink outdoors the convention community. A breakdown of connectivity for the above path is proven under:
The throughput quantity above is vital to this investigation. The Entry Factors (APs) for the Hibiscus 3610 ballroom had a median throughput of round 174 Mbps. Reviewing AP logs, we discovered that 92 customers have been related to the identical AP from which the take a look at was run. Dividing the 174 Mbps by 92 offers a median throughput according to the 1.7 Mbps proven above, so the poor connectivity was pushed by oversaturation of person connections on this space.
The Hibiscus 3610 room and different brokers in a close-by hallway persistently had the worst connection among the many convention rooms, as proven by our agent polling outcomes.
Whereas there have been limitations within the quantity of bandwidth obtainable for the convention on the whole, the information above suggests extra of the obtainable AP and bandwidth assets needs to be allotted to the Hibiscus 3610 ballroom and adjoining hallways for future convention topologies, which was shared with our Community Gear associate.
Meraki Methods Supervisor, by Paul Fidler and Connor Loughlin
Our eighth deployment of Meraki Methods Supervisor because the official Cellular Units Administration platform went very easily, and we launched a brand new caching operation to replace iOS units on the native community, for velocity and effectivity. Going into the occasion, we deliberate for the next forms of units and functions:
- iPhone Lead Scanning Units
- iPads for Registration
- iPads for Session Scanning
We registered the units prematurely of the convention. Upon arrival, we turned every gadget on.
Then we ensured Location Providers enabled, at all times on.
As an alternative of utilizing a mass deployment know-how, like Apple’s Automated Machine Enrollment, the iOS units are “ready” utilizing Apple Configurator. This consists of importing a Wi-Fi profile to the units as a part of that course of. In Las Vegas, this Wi-Fi profile wasn’t set to auto be part of the Wi-Fi, leading to the necessity to manually change this on 1,000 units. Moreover, 200 units weren’t reset or ready, so we had these to reimage as effectively.
Black Hat Asia was totally different. We took the teachings from Black Hat USA 2023 and coordinated with the contractor to arrange the units. Now, should you’ve ever used Apple Configurator, there’s a number of steps wanted to arrange a tool. Nevertheless, these could be mixed right into a Blueprint.
For Black Hat Asia this included:
- Wi-Fi profile
- Enrollment, together with supervision
- Whether or not to permit USB pairing
- Setup Assistant pane skipping
In Meraki Methods Supervisor, we managed the purposes by the assigned use, designated by Tags. After we got here in on the primary morning of the Briefings, three iPhones wanted to be modified from lead scanning within the Enterprise Corridor, to Session Scanning for the Keynote, so the attendees might fill the corridor sooner. Reconfiguring was so simple as updating the Tags on every gadget. Moments later, they have been prepared for the brand new mission…which was necessary because the Keynote room stuffed and needed to go to an overflow room.
We additionally have been capable of affirm the bodily location of every gadget if wiping was required resulting from loss or theft.
When it was time for the attendees to register, they simply displayed their QR code from their private telephone, as acquired in electronic mail from Black Hat. Their badge was immediately printed, with all private particulars secured.
This goes with out saying, however the iOS units (Registration, Lead Seize and Session Scanning) do have entry to non-public data. To make sure the safety of the information, units are wiped on the finish of the convention, which could be accomplished remotely via Meraki Methods Supervisor.
Content material Caching
One of many largest issues affecting the iOS units in Black Hat USA 2023 was the speedy have to each replace the iOS gadget’s OS resulting from a patch to repair a zero-day vulnerability and to replace the Black Hat iOS app on the units. There have been a whole lot of units, so this was a problem for every to obtain and set up. So, I took the initiative into wanting into Apple’s Content material Caching service constructed into macOS.
Now, simply to be clear, this wasn’t caching EVERYTHING… Simply Apple App retailer updates and OS updates.
That is turned on withing System Setting and begins working instantly.
I’m not going to get into the weeds of setting this up, as a result of there’s a lot to plan for. However, I’d recommend that you simply begin right here. The setting I did change was:
Location and Jailbreak detection
One factor that we haven’t spoken about in a while is Jailbreak detection and Location. There are lots of components that we get again from a tool, however two of them, Location and Jailbreak have to be retrieved from a tool utilizing a supplemental utility: On this case, the Meraki Methods Supervisor agent.
HOWEVER, these can solely be retrieved from the gadget if the applying is working within the background. If the gadget has been rebooted, or the applying terminated, then we don’t get something.
One of many different painful, however comprehensible, elements of MDM is you could’t launch an utility distant on a cellular gadget…. However you possibly can!
On each Android and iOS, there’s a functionality referred to as Kiosk or Single App mode: Use instances for this are usually unattended units, like in eating places, or scanning units like supply drivers, and so forth. And when sending the command to the gadget to enter kiosk mode will launch the applying. You can even ship a command to take away kiosk mode from the gadget too. The beauty of this final level is that the applying stays in focus and open!
So, the opposite functionality that utilizing Meraki Methods Supervisor offers us is the flexibility to schedule settings. Due to this fact, we will activate kiosk mode in the midst of the night time and take away it an hour later.
To make sure that this doesn’t influence the registration employees, we will go one step additional: after we’ve launched Meraki Methods Supervisor, an hour later we will relaunch the registration utility, Swapcard Go.
![](https://storage.googleapis.com/blogs-images/ciscoblogs/1/2024/05/Screenshot-2024-05-14-at-5.02.38 AM-1024x520.png)
![](https://storage.googleapis.com/blogs-images/ciscoblogs/1/2024/05/Screenshot-2024-05-14-at-5.04.54 AM-1024x531.png)
Systematic ThousandEyes Agent Deployment
ThousandEyes has been a success at Black Hat. At an occasion the place understanding instantly the place points lie within the community and past to make sure an awesome convention is paramount, the visibility ThousandEyes offers is unbelievable. On condition that, and the complexity of the community right here, and on condition that now we have a Mac Mini deployed for caching software program updates, as we’re utilizing Meraki Methods Supervisor (SM) for different functions, I believed I’d take the chance to deploy the ThousandEyes Agent utilizing SM.
The opposite cause is that, while now we have a substantial quantity of cloud and enterprise brokers, we had no endpoint brokers deployed. Nevertheless, issues are by no means that simple with software program deployment, primarily as a result of it’s essential to provision / configure software program as soon as deployed. On cellular units, that is easy, both utilizing settings payloads, or through the use of Managed Appe Config to configure an app.
On desktop, utilizing MDM, we will usually use issues like Managed Plists to do the identical factor, however the TE agent does NOT assist this. As soon as put in, we should name the agent with a string.
So, to realize all this, we will package deal the agent and command right into a package deal utilizing a command line utility on the Mac referred to as PKGBUILD (extra particulars right here).
I additionally used a information I’d written for the Meraki Group, obtainable right here.
Information of observe:
The Postflight:
#!/bin/bash
# this identify will change with every model of the agent
installer -pkg /tmp/Endpoint Agent-x64-1.193.1.pkg -target /
/Functions/ThousandEyes Endpoint Agent.app/Contents/MacOS/te-agent –register “YOURUNIQUESTRING”
exit 0
The command to construct the package deal utilizing PKGBUILD
Extra particulars right here or watch the video.
Repurposing of Units for the following present
We have been requested if there was something we might do to depart the units as they have been for the following present. After cautious consideration, we determined that we might go away the units in a state that was amenable to everybody. The foremost requirement was leaving the Swapcard Go app on the gadget. However, because the app is provisioned for every present, it’s fairly the method to take away configuration after which re-add it….
So, the opposite factor to notice is the choices that now we have when putting in (and eradicating) an utility on a managed iOS gadget:
Take away with MDM is the fascinating one, because it permits us to, somewhat than WIPING the gadget on the finish of the present, to take away administration, together with any apps and settings, and their corresponding knowledge.
The downside with that is that this was by no means a requirement firstly of the present. So, we now want a course of in a selected order to facilitate this…. As that is for under a handful of units:
- Deprovision the app from units by unscoping the applying in Meraki Methods Supervisor
- Wait to see this command has accomplished throughout all units
- Reprovision the app utilizing MDM once more, however with this being a brand new app set up, it’s going to enable the OS to maintain the app in situ after an unenrollment
- Wait till accomplished
- Unenroll the gadget
Area Title Service Statistics, by Christian Clasen
Since 2018, now we have been monitoring DNS stats on the Black Hat Asia conferences.
The historic DNS requests are within the chart under.
With over 18.2M DNS requests made, we had essentially the most up to now at an Asia present. We made visibility developments on the earlier yr’s Asia convention. Previous to Asia 2023, we have been permitting attendees to make use of their chosen DNS resolvers over our assigned inside Umbrella Digital Home equipment. In coordination with Palo Alto Networks (the convention Firewall supplier), we started intercepting and redirecting DNS queries for different resolvers, to pressure decision via the Umbrella gear. Whereas that is solely efficient for plain-text DNS queries and never encrypted protocols like DNS over HTTPS, it never-the-less dramatically elevated visibility as evidenced by the numbers within the accompanying charts.
The Exercise quantity view from Umbrella offers a top-level stage look of actions by class, which we will drill into for deeper risk looking. On pattern with the earlier Black Hat Asia occasions, the highest Safety classes have been Malware and Newly Seen Domains.
In a real-world atmosphere, of the 18.2M requests that Umbrella noticed, over 2,000 of them would have been blocked by our default safety insurance policies. Nevertheless, since it is a place for studying, we usually let every part fly.
We additionally monitor the Apps utilizing DNS, utilizing App Discovery.
- 2024: 4,327 apps
- 2023: 1,162 apps
- 2022: 2,286 apps
App Discovery in Umbrella offers us a fast snapshot of the cloud apps in use on the present. Not surprisingly, Generative AI (Synthetic Intelligence) has exploded over the earlier yr as a prime utility.
Umbrella additionally identifies dangerous cloud purposes. Ought to the necessity come up, we will block any utility by way of DNS, similar to Generative AI apps, Wi-Fi Analyzers, or the rest that has suspicious undertones.
Once more, this isn’t one thing we’d usually do on our Normal Wi-Fi community, however there are exceptions. For instance, from time to time, an attendee will study a cool hack in one of many Black Hat programs or within the Arsenal lounge AND attempt to use mentioned hack on the convention itself. That’s clearly a ‘no-no’ and, in lots of instances, very unlawful. If issues go too far, we are going to take the suitable motion.
Throughout the convention NOC Report, the NOC leaders additionally report of the High Classes seen at Black Hat.
General, we’re immensely happy with the collaborative efforts made right here at Black Hat Asia, by each the Cisco staff and all of the companions within the NOC.
Black Hat USA will likely be in August 2024, in Las Vegas. Christian Clasen will lead the Cisco staff within the NOC, so comply with his weblog to see if what occurs in Vegas, stays in Vegas.
Acknowledgments
Thanks to the Cisco NOC staff:
- Cisco Safety: Christian Clasen, Shaun Coulter, Aditya Raghavan, Adam Kilgore, Patrick Yong and Ryan Maclennan
- Meraki Methods Supervisor: Paul Fidler and Connor Loughlin
- Further Assist and Experience: Adi Sankar, Robert Harris, Jordan Chapian, Junsong Zhao, Vadim Ivlev and Ajit Thyagarajan
Additionally, to our NOC companions NetWitness (particularly Iain Davidson and Alessandro Zatti), Palo Alto Networks (particularly James Holland and Jason Reverri), Corelight (particularly Mark Overholser and Eldon Koyle), Arista Networks (particularly Jonathan Smith), MyRepublic and the whole Black Hat / Informa Tech employees (particularly Grifter ‘Neil Wyler’, Bart Stump, Steve Fink, James Pope, Michael Spicer, Jess Jung and Steve Oldenbourg).
About Black Hat
Black Hat is the cybersecurity business’s most established and in-depth safety occasion collection. Based in 1997, these annual, multi-day occasions present attendees with the most recent in cybersecurity analysis, growth, and traits. Pushed by the wants of the group, Black Hat occasions showcase content material straight from the group via Briefings displays, Trainings programs, Summits, and extra. Because the occasion collection the place all profession ranges and tutorial disciplines convene to collaborate, community, and focus on the cybersecurity matters that matter most to them, attendees can discover Black Hat occasions in the US, Canada, Europe, Center East and Africa, and Asia. For extra data, please go to www.blackhat.com. See the press launch for Black Hat Asia 2024.
We’d love to listen to what you suppose. Ask a Query, Remark Under, and Keep Linked with Cisco Safety on social!
Cisco Safety Social Channels
Share: