Phone Number +1-202-802-9399 (US)

Thycotic PAM, IT and Cyber Security Podcast
Listen on-demand

401 Access Denied Podcast

Welcome to the 401 Access Denied Podcast, where we dissect what’s really going on in today’s world of cyber security. Topics range from finding a job in cyber security, to dealing with insider threats, to going inside the mind of a hacker, and more.

Bi-weekly, Thycotic’s ethical hacker Joseph Carson and the cyber security training experts from Cybrary will share their insights along with our special guests.

Want to give input on our next cyber security podcast? Give us your topics

Subscribe or listen now on your favorite podcast app:
Apple | Spotify | iHeartRadio

Voted "Best Cybersecurity Podcast" in the 2021 Cybersecurity Excellence Awards
Cyber Security Excellence Awards 2021

Thycotic produces this podcast in partnership with Cybrary, the cyber security and IT career development platform.

401 Access Denied

Episode 20

The Latest from the SolarWinds Sunburst Breach

EPISODE SUMMARY

In this week’s 401 Access Denied episode, we’re giving you the latest on the recent SolarWinds Sunburst breach that affected FireEye, the US government, and thousands of other organizations. This incident has the potential to be the biggest supply chain attack in history. The team discusses what the investigation has uncovered so far and how these lessons can be applied to make the cyber world a safer place.

Special guests Jonathan Meyers, Principal Infrastructure Engineer and Head of IT at Cybrary, and Terence Jackson, Chief Information Security & Privacy Officer at Thycotic, join Mike and Joe today.

powered by Sounder

Free Tools

Take the first step to protecting your privileged accounts with Thycotic educational resources and free PAM software products.

→ See All Privilege Management Tools

Secret Server Icon

Secret Server Free

The perfect password management starter tool. 10 Users, 250 Secrets.

Icon - Audit

Password Security Policy Template

Icon - Project

Privileged Account Discovery for Windows

Icon - Test

Customizable Incident Response Template

Icon - Virus

Weak Password Finder for Active Directory

Joseph Carson

  • Chief Security Scientist at Thycotic
  • Over 25 years' experience in enterprise security
  • Author of "Privileged Account Management for Dummies" and "Cybersecurity for Dummies"
  • Cyber security advisor to several governments, critical infrastructure, financial and transportation industries
  • Speaker at conferences globally
mm

Terence Jackson

  • CISO at Thycotic
  • Over 17 years of public and private sector IT and Security experience
  • Responsible for protecting Thycotic’s information assets
  • Leads a corporate-wide information risk management program
  • Identifies, evaluates and reports on information security practices, controls, and risks in order to comply with regulatory requirements and to align with the risk posture of the enterprise
mm

Mike Gruen

  • Cybrary VP of Engineering / CISO
  • Manages Cybrary’s engineering and data science teams, information technology infrastructure, and overall security posture
  • 20+ years of experience developing and overseeing the implementation of complex, secure, and scalable software solutions and products
  • Previously served as VP of Engineering and VP of Product & Platform at RedOwl
  • B.S. in Computer Science from the University of Maryland at College Park

Joseph Carson:
Hi, everyone. My name is Joseph Carson, Chief Security Scientist here at Thycotic, and I'm really excited to bring you our latest show of 401 Access Denied. We have some fantastic guests for you today who we're going to be really getting into discussing a very hot topic, and something that may have consequences for years to come. And I'm joined here with my awesome co-host of the show, Mike Gruen. You want to give a little bit of update into yourself and how amazing you are in a new studio?

Mike Gruen:
Yeah, thanks. So yeah, Mike Gruen, VP of Engineering and CISO here at Cybrary. And today, we have a couple of guests from our respective companies. I'll start with Jonathan Meyers. Jonathan, if you want to give a quick intro?

Jonathan Meyers:
Hey, I'm Jonathan Meyers. I'm the Principal Infrastructure Engineer here at Cybrary and also the head of IT. So, yeah. Glad to be here.

Mike Gruen:
And our other guest, Terence?

Terence Jackson:
Hi, my name is Terence Jackson, and I'm the Chief Information Security and Privacy Officer at Thycotic.

Joseph Carson:
Awesome. And we're here to have a really intriguing discussion. Let's see where this goes today, but we're her to talk about SolarWinds and this Sunburst incident that happened going back into early December. Where really basically, when we started finding out was when FireEye disclosed that they had become the victim of a security incident. And for me, FireEye are one of the best in the industry at what they do. They have some awesome instant responders. They have amazing malware analysts, and for them to become a victim of an incident was quite... well, it's never surprising because ultimately, all companies will have security incidents, but what was kind of surprising here was how quick FireEye worked to disclose it and announce it and make it available, and through their investigation of course, they found out that it came through a supply chain attack, ultimately kind of going back into SolarWinds. And as more information was made available, we started seeing that this has the potential for me of being one of the biggest software industry supply chain hacks to date in history, with over 18,000 initial victims that were victims of that stage one campaign portion. And we'll get into a bit more details into really what happened on the back line and give you the latest information we have today.

Joseph Carson:
I mean, as we're recording today in January, which is right now the 21st of January, this is where we're recording, every day there's a new piece of information being made available to us. And I think already of this week, I'm seeing I think it was the fifth or sixth biggest piece of news announcements.

Joseph Carson:
And I always go back to one of the people I respect in our industry being Brad Smith, I've known for quite a number of years. He was the President of Microsoft, and he's stated that this is probably a mass indiscriminate global assault on the industry that we've ever seen before. And he made that announcement at CES Technology Trade Show a few weeks ago in the keynote. And I think this has many major consequences that we're seeing, and just kind of for the audience or cohosts here, do you want to provide some of your view of what you're seeing? What's the potential and what's the risk we're having here, in regards to this latest incident that we're seeing? I'll pass over to Terence to give the first comment.

Terence Jackson:
Sure. I've been saying for years you're only as strong as your weakest link, and SolarWinds is a very popular network monitoring performance tool. Various companies from your Fortune 100 companies to half of the federal government has SolarWinds deployed in their infrastructure. So, I think this attack was very methodical in how it was executed, and exploiting weaknesses in the software development pipeline. And we always talk about in the industry dwell time, there was massive dwell time around this. And as you mentioned, new news comes out every day, every week.

Terence Jackson:
I think that's going to continually be the case for the remainder of this year, because we don't know what other Trojans have been left behind or deployed that are just in a sit and wait state. But I think this was a wake-up call for all of us security practitioners to reiterate trust but verify, especially when it comes to our supply chains. Each of our companies relies on some type of cloud or SAAS-based infrastructure to operate. And we can't control a lot of the underlying infrastructure in that, and obviously SolarWinds pushed out an update and the company's like, "Hey, we tell everybody to patch, patch, patch." But we just have to ensure that we're doing at least as much due diligence as we can when we're vetting and evaluating new vendors. But also, security's a team sport, so I think this is also an opportunity for us to get tighter as a community and just help each other out when incidents like this occur.

Joseph Carson:
Absolutely. SolarWinds are a friend in the industry. I mean, we're all friends. We're all here to achieve the same goal. We're all here to make the world a safer place. We do get competitive, we're trying to sometimes get the same customers and so forth. But at the same time, our goal is to make the world a safer place, in regards to the digital place that we actually do our work in and everyday lives. And for me, I think SolarWinds and the customers of SolarWinds are not just the victims here. It's the entire software industry. This is the victim. I'd just like to get Jonathan, some of your insights or views and what your thoughts and opinions are of the current incident today.

Jonathan Meyers:
Yeah. I think it's pretty interesting. I think we've kind of started to see murmurs... Well, bubbling of supply chain stuff starting to happen in the industry, right? Especially with a bunch of companies that use the Node frameworks and those dependency tree Ls and starting to see... I think this just kind of brought to light a lot of the stuff that people were kind of thinking that could happen. It's like, "Oh, well, we used these 12,000 dependencies in our UI app for Node," and it's like well, what if one of those is compromised? And I think this is effectively that on a much grander scale for the most part. But my kind of first take on a lot of this stuff is how could you... I don't even know how a company that has SolarWinds could have prevented this, right?

Jonathan Meyers:
You would have trusted SolarWinds. Your due diligence is like, "Oh, it's SolarWinds. Cool. Let me see all your security creds." I don't know where you would have been able to even start to think about, "How do I stop this?" Especially teams are just the largest teams ever, that just have an entire body to throw at a software vendor for a week and be like, "Hey, let's check out the security update." But I also from the government side of the house, and I remember installing software on CD-ROM drives and scanning all of those, and doing all of that type of stuff. And I still don't think that would have caught of any of this, doing manual scans of these packages.

Jonathan Meyers:
So it's I think what you said, it's very much an industry, we're all on the same team. How can we start to iterate on this faster and kind of predict these things? And stuff like that, so.

Joseph Carson:
Absolutely. I think this needs an industry revolution to happen for us all to come together and find a... analyze this in as much depth as we possibly can, and really come together and find a way forward. I've been in the patch management business for 20 years. And for me, it's broken our patch software update process. This whole incident has raised that to the surface, and if we kind of go back to showing how sophisticated this was and just going over what Terence had mentioned, this was really like a surgery that was very, very sophisticated, very well-planned, very well-executed. Even dating back, I mean, this has been several years in the planning, only possibility of being either a nation state or a nation state-backed group could have been really possible to pull this off.

Joseph Carson:
I always look at what I'm doing as a response, I get into the motives and the intentions behind it. And there's many motives here. What we're seeing is there's financial motives that appears to be happening in the news today, that they're now selling parts of the source code off of the victims that they've had. So there seems to be financial participation here, intellectual property theft as well. You've got espionage, potential pre-staging of preemptive attacks in the future. There's so many possible motives in the background here, it's really hard to really find out what was the true... I always find that there's one true motive, and I think we're really getting to the point where that hasn't really come to the surface yet. When we get into looking, to your point Jonathan, I don't think anyone could have prevented this. I think you once you became a victim, you could react to it and you could see it later, maybe have some controls in place, but stopping it from happening... It goes back to I think the first main registration of the C2 was back in 2018.

Joseph Carson:
Then you get into in 2019 they're already getting access to the build repo in SolarWinds and started just analyzing, intelligently gathering information about how SolarWinds actually develop and build and compile code. Just watching and learning and watching and learning and perfecting it, to the point is that when they eventually actually put the Trojan in place in the malware, it was actually to the point where it actually looked like SolarWinds code. A developer looking at that would not have been able to tell the difference, and then of course getting digitally signed, and actually making sure it was done in run time during only the compilation process. And then ultimately, like most companies, once you get that signed build, you want companies to update it. And it was IT who downloaded it into those companies, and distributed it to the servers. It was IT that was actually in the process of that, and sitting for two... It was 12-14 days of doing nothing, trying to stay... obfuscation, staying hidden. This was stealthy in design. So for me, it was done to basically very detailed surgery. This has been years in the planning, and potentially probably using some of the blueprints of previous attacks.

Joseph Carson:
Whether it being coming off the back of Stuxnet or some of the things that we saw in the... not ... of course that was a software update attack as well. And even go back into I think it was 2004, when the big sport was in Greece or somewhere around that time where they also used a supply chain attack in the ... equipment. For me, this has probably taken a lot of what we've seen over the years, but really perfected it. So Mike, when you're listening to that perspective and you being involved in a lot of software development, how can companies really kind of respond to that?

Mike Gruen:
Well, before I get into that, I think just sort of some things that I thought were really interesting about the attack overall and from that software engineering perspective, they didn't just go straight to it. They did a test, they did test runs, made sure they'd be able to fly under the radar. I'm happy to be told I'm wrong, but I think we might be giving them a little bit too much credit with regard to how much time they've been planning this specific attack.

Mike Gruen:
I think if I was doing this type of activity, I would have hundreds of command and control domains registered that I'd be registering well in advance of maybe ever needing them, just so that they're there, the DNS records are there. They've been sitting there for years dormant, just waiting for, "Hey, I finally found a vector, what can I use that I've already sort of set up?" And I think from a motive perspective as you said, there's so many possible motivations that I think if I was an attacker, if I'm a nation state, I probably have all of those motivations. So I don't know that we can sort of focus in on what was the specific one. I think it was, "Hey, let's go for the broadest attack that we can or find a target that gives us the most opportunity."

Mike Gruen:
I mean, that's sort of how I would go about doing this. So I think when you're looking at why SolarWinds and who, it's like well obviously, they're a great target because from there, they're a great launching pad into so much collateral... what we said was collateral damage, whether it was Microsoft or whomever who just gets a hop or skip or jump from there into so many other places.

Joseph Carson:
Yeah, and just to your point Mike, I think you mentioned an important piece there for me, is that we all have to be looking for these attacks in our environments. We have to be proactively right now looking, and even if you're not a customer of SolarWinds, you should be concerned and looking for this. Because what we're finding is that you might be a customer of a customer of SolarWinds. And this flows down. This is a domino effect, is that if the supplier of you is a customer, who you're dealing with, and that might be a cloud customer, it might be a SAAs-based customer, you should be concerned and you should actually be proactively looking to make sure you're not a secondary victim or even a third victim of this.

Joseph Carson:
Because this has the consequences, because we're all interconnected. We're all an industry that many companies out there have so many vendors, and their vendors they deal with are also working with many vendors. So this is all a massive interconnected web of different suppliers and third parties in the source code and libraries that the consequence of this is potentially, everyone could be a victim at some point.

Mike Gruen:
Yeah, I mean, I think the web of this is interesting, right? Because even if you're not a SolarWinds customer, let's just take the Microsoft route. Microsoft was impacted. Who doesn't have someone in their supply chain, even if we're not directly using Microsoft, which we're not, there's no way that Microsoft is not involved somehow in something that we do. And so I think that the potential is almost every tech company is impacted by this, and any company that uses technology.

Joseph Carson:
Yeah. Terence, I'd like to get your thoughts on that as well. Microsoft being collateral damage, in your thoughts?

Terence Jackson:
Yeah. A couple of things. The low and slow approach that this attack took should be something that we really pay attention to. Because if you look at the victims that we know about and the 18,000 number, FireEye being one, these guys have some of the most sophisticated tools, technology, smart people, and they missed it. So from that perspective, I think there's some areas of opportunity for the industry on where we can all improve. ML, AI, for the last five, six, seven years, next gen this, and to my knowledge, all of that failed. I mean, all of it failed.

Joseph Carson:
Well, that's when you should listen to one of our older episodes.

Terence Jackson:
So I think just pivoting back to Microsoft, Microsoft's been kind of all over this early on, and some of the things that they did by revoking domains and getting those seeds taken down, and pushing out IOCs to their customers that were using their end point security products, I think it's a good model of security being a team sport and them making the intel readily available to their customers, and even to the point now they're enabling certain features by default to help customers better protect their environments. And I think that's a model that as an industry, being the CISO of a cybersecurity product company that we can all do better, and be more responsible in enabling our customers to respond to threats not if, but when they happen. But yeah, I absolutely think Microsoft was collateral damage.

Terence Jackson:
I think that they ended up in a bunch of environments that were using Office365 and they had SAML and MFA configured. And they got curious, and they were like, "Oh, this actually works." And they waited, it worked again. And then they did that, "Who am I? Where am I? Okay, it's Office365 in play, let's go forth." And this is not a knock on any MFA vendor, Duo and others, but if they were using SAML tokens, it was kind of game over, so.

Joseph Carson:
Single sign-on out the window.

Terence Jackson:
Exactly.

Joseph Carson:
Single sign-on for the attacker, that's what you're looking at, so.

Terence Jackson:
Exactly. So I think there are a lot of takeaways for us as security practitioners to look at. And obviously when this happened, everybody... the board and all the execs and everybody had the same question, "Are we a SolarWinds customer? And how secure are we?" And I think everybody is going through that process of re-evaluating controls and looking at intel that's available. FireEye's done a very good job in releasing reports. They released one yesterday, as well as Microsoft. But threat intel I believe is becoming more important to be incorporated into our security programs. I've seen a fair amount of email traffic from threat intel vendors nowadays, in making that a part of your program. Which we already had, but I think it's a pivot point for the industry, the people, process, and technologies.

Terence Jackson:
I think we've gotten processes down. The people problem, we're solving but the tech is not a magic bullet in any stretch of the imagination. And I think this kind of turned that theory upside down, that what if you're a security vendor? Or the tool that you use to monitor the status of your environment is compromised? What happens then? And unfortunately, I don't think there's a clear-cut answer to that, but it definitely involves all three. The people, process, and technology.

Joseph Carson:
Absolutely. I just want to bring Jonathan in, two kind of, or at least for you to get your thoughts on, is since this did compromise SAML token forgery which is a big concern for me because it does allow not only the attacker to basically get access to the victim's own infrastructure, but also pivot so many different areas, whether being into cloud applications, housing applications, SAAS, anything that basically organizations have really put in their service provider in regards to their single sign-on. And the second part of that as well is that this did leverage a digital certificate. Can we trust digital certificates going forward? So I'd like to get your thoughts on those two items, and anything else that's on your mind as well?

Jonathan Meyers:
Yeah. I don't know if this has been confirmed or denied, but some of the reports coming out of FireEye was how they originally caught it. And one of them is these guys requested a multifactor token, and it triggered an alarm because there was no employee that was in their system that had been assigned this token, and so I would say that tokens did do some sort of slowing down. Right? Because once they had that, I think if they had that full token that was authorized, I think it'd have been game over. So I think that's pretty interesting, but that's weird, and I think the way they got caught was still a human that sat there and said, "Wait..."

Jonathan Meyers:
Right? Because if somebody wasn't monitoring that and it just sat there in the unassigned bucket, and was still authorized, I think that's a very, very interesting sort of play. And then back into the SAML stuff, I think it's a weird problem to solve for because it's just like, "Oh, they have the keys to your house." It's like, "How do I do this?" It starts to go back that I think we're going to start hopefully maybe as an idea, you're migrating to this person as an identity, and there's no longer... right? I think one of the weird things is as IT admins, you all have multiple sets of credentials that are least privilege and things like that, and it's like, I think that starts to get into this weird thing, because there's not this one to one match between a person that works for your company and the permissions of the things they can do. And I think this kind of highlights that, as like, "Oh, there's a bunch of accounts out there that don't actually match to a human, and they're out there doing things in an environment."

Jonathan Meyers:
How do we start to ensure that it's a one to one relationship? And I think things like physical security keys, where it's like you got to push a button physically, that's like, that lets you in. I think that starting to kind of implement those on much higher levels of access and things like that starts to get to that point. The certificate problem is a huge problem. I think we've slowly been moving that way, right? I think certain things in the certificate world are starting to move that way.

Jonathan Meyers:
I think it's just very slow rolling, with the whole, "Let's encrypt," and the ability to automatically generate certs and things like that, I think you can start to eliminate a lot of it if that infrastructure kind of gets baked in. So now you're turning certificates every day or 12 hours or six hours, and there's a method to ensure that as soon as a key has been deemed compromise, that there's a way to revoke it without breaking everything in the system and down the lines. And I think kind of getting to a point where we're rotating things a lot faster than we thought about in the past is an interesting strategy moving forward for that.

Mike Gruen:
Yeah, I think with the ... at some point, there's still some key. There's something, up chain. You always have that potential of, "Well, okay, so these certificates are rotating all the time." It's going up the line, and I think that's where it gets trickier and trickier and Joe and I have talked about things that people are good at and things that people are not good at when it comes to passwords and having an identity so that I don't have to put in a password, but it's more of a cert-based system, but then this sort of gets into almost the exact opposite of that. Sometimes, a human in the system is the thing that's going to pull the kill switch because they can recognize this just doesn't make sense.

Mike Gruen:
This isn't right, like what happened at FireEye, right? That's what ultimately happened. And I don't know, there's no amount of AI or ML that's going to solve those problems. And ML and AI, that type of stuff, that'll find it after it's already released. That'll sort of see it in action and be like, "Oh, that looks unusual," but at that point, it would have already been too late again. So, I don't know how we sort of deal with that.

Terence Jackson:
Right. To that point, AI and ML in its current iteration is nothing but models that have been trained.

Joseph Carson:
Automation.

Terence Jackson:
Exactly, it's automation. It is, but it isn't. But I'm curious what your perspectives are on now, because the industry for the last 10 plus years has been vulnerability management, patch, patch, patch, patch, patch. So now, what do we do? Are we...?

Mike Gruen:
I'm buying a 1970s car and moving to Montana. That's my solution.

Joseph Carson:
That's the challenge, because we do software updates and security patches the exact same way. The process is exactly the same, and the delivery mechanism is exactly the same. And I always had this fun terminology, is that it used to be when Microsoft released their patch, it comes out Tuesday, you would test it Wednesday, pilot it on Thursday, roll it out Friday, and then you'd burst into tears on Saturday because it failed. And then you're using your Sunday for rolling back, and then you're testing it again on Monday, the new patch. And then you're rolling it out Tuesday, and then it's repeat. But we treat the software update, new feature functionality follows that same process and path, and it's the same signature or the same compilation process.

Joseph Carson:
And we want people to apply their software updates fast. We don't want them to delay, because there are already exposed vulnerabilities-

Mike Gruen:
You mean their security updates?

Joseph Carson:
Security updates, correct.

Mike Gruen:
And it's funny, because dev sec ops is all about like, "Let's treat our security updates the same way we treat our software updates in a lot of ways," right? If we want to shift software developers to being more responsive to security problems, then what we're doing is we're shifting those identifications and treating them just like any other bug. Right? We shifted into the testing, and to all sorts of things, and the CSED and to their IDE so that like, "Hey, while you're fixing this other thing, update the dependency on this package that we use, because there's a security update." Fundamentally, there's no difference between a security update and a bug fix.

Mike Gruen:
We can get into new features and new whatever, but fundamentally, they're the same thing. And so, I think we do want to continue to move them together. And as someone who had to deal with enterprise software releases, the more you can sort of put in the release, the better. You can't expect a customer to... and Jonathan knows, because he was on the installing end of this. We had customers that were running the same things for months because they weren't interested in those new features or whatever. For enterprise software, you want to have fewer releases. And so, the more you can sort of bundle together to get those companies to update and upgrade is the right thing. And obviously, security patches like, "Oh, this is a vulnerability that needs to be patched immediately," falls outside of that. But I think that's where it's going to be a struggle.

Joseph Carson:
Yeah. We talk about zero trust, and I hate the terminology zero trust. It's not my biggest favorite term. I prefer looking at it from building trust, because ultimately, we want to establish a trust framework, so it's all about building trust. And for stuff, updating patch, security updates and all those processes, I fundamentally am concerned that right now, how can you trust the next update from another vendor? And how long do you need to test it? Do we need to put it in sandbox now for two weeks, and month? How long do we have to delay that in order to be able to trust it? And this gets into that whole scenario. What are we doing? Do we need to get into having more controls and more validation and continuous verification over that process? Because right now, I think what we're going to start seeing is the fallout of this incident, we're starting to see that Microsoft's now a victim and their source code is now exposed.

Joseph Carson:
We see Cisco's out there, Malwarebytes have just recently announced and Symantec have now released their latest findings of both the Malware Teardrop and Raindrop, which is awesome. So there's been right now, so many malware variants have been involved in this particular incident that it gets to the point where it's really, "Where do we go from here?" All of those source codes have now been exposed. The attackers are probably now looking for vulnerabilities in those, looking for exploits. And then the update again is the same process, is to download and update and patch. So we're in this kind of basically mouse wheel or mouse trap, in a kind of continuous process. Where do we really need to evolve? Where do we need to get to? And I think this is where the industry needs to come together. This must be a collaboration effort for me, because I think we're going down this repeat path again in order to solve the problem of yesterday.

Mike Gruen:
I mean, I think risk is an important factor, right? Just forget about what we do moving forward, but what we do today. When we look at some new version or an update, I do look through it and sort of decide, how important is it that we do this timely, right? And if it's a security patch, obviously it has a higher risk. The longer we wait, the more we're exposed. So obviously, we need that rush, but get this through faster. And things that are just feature function can take a... We can have a longer burn and we can make sure and we can test. To your point, and we started talking about this a little bit was I think we need to potentially grade things or provide that risk profile so that me, I'm on the tail end. Nobody depends on Cybrary's software, right? We are the leaf on the tree. We depend on everybody else leading up to us. And so, when we go to make those decisions, what we need in order to decide whether or not we want to upgrade is what's the risk?

Mike Gruen:
What's the potential, and how much risk are we willing to take on for us and our customers? And so yeah, security things are going to have a higher precedence, and just features, we should be testing longer. And I think we need a better way of identifying that, communicating that as part of the patch process for automation. Because right now, we could totally automate, just update. Whenever there's a new version, just update and run it through some smoke tests or whatever, and if the house doesn't burn down, let's just ship it. It needs to be built into the automation. There needs to be flags, there needs to be some way of indicating a risk for a file of some sort. I don't know. That's just my thoughts off the top of my head.

Joseph Carson:
And I'd like to bring Terence and Jonathan in, I've got a question on your side as well, is that one of the things that the original Sunburst malware software, it was staying there, it had the back door. What was kind of impressive about that is of course, once the payload had been delivered in those stage one victims, where it started doing this enumeration phase of gathering things like system information, confidential details, the administrator's registry details, it started kind of doing that enumeration of what environment am I in. What was impressive was is that this actually put itself into what was called an Orion Improvement program. It masqueraded as telemetry type of data, and then sending it back in through HTP, and it did really... and you used stenography. Stenography, it's hard to say that word properly. It's a tongue-twister.

Joseph Carson:
But it hid itself in basically standard data configuration information and then through each piece, send it off to the command and control, which was located in the US. So sometimes, when you're using threat intelligence, you see it going off into countries where you may not do business and that's always a flag, or you start seeing new traffic that's outside of what you should be monitoring. When your data's been stolen from, or your information is being sent to command and control, what things do you see, what can we do there to prevent our data being taken out and also to get that visibility over potential malicious network traffic?

Terence Jackson:
That's a great question, man. One of the things I would say that my team currently does, we have rules in our SIM around spikes in encrypted traffic that's going out, and just anomalous activities around increases in network traffic overall. And this is one of the areas where I would say ML or AI actually does help us, because it's able to evaluate that looking at standard deviations over an amount of time. And it does detect minute spikes that a human would otherwise ignore, so-

Mike Gruen:
I mean, just to but in there, I would just program for that. I would just go for a slow burn. If I know what the algorithms are, then I just go lower and slower, and your AI, your ML won't pick that up. It won't be a spike, it'll look like any other sort of system naturally coming online. We'll just sort of have a slight increase.

Joseph Carson:
Which is what happened here. Is that they masqueraded in telemetry data, and also made it look authentic in HTPs. And it was hidden in plain sight, which is the challenge here.

Terence Jackson:
Right. Well, to that point-

Terence Jackson:
... we actually paired that with domains never seen before. Which both of you will probably be like, "Wow, that's probably way too noisy." But again, the solution that we have actually boils that down and does a layer of analysis on that that's actually curated by a human on the back end. So it's an outsourced service, but at the end of the day, if you're in the cross hairs of a nation state, it's basically hold on and hope that you can detect it sooner rather than later and you don't have a nine-month dwell time. But it's still back to a lot of the defense and depth approach. I call it like an onion man. It's layers, there won't be a magic bullet, but I believe there are things you can do in your environment to detect things quicker. But like I said, if you have somebody determined and they're going low and slow, it's tough.

Terence Jackson:
I mean, I wish I had an answer because I probably would go start another business. I don't, so if you guys want to talk after this. I think there's a blockchain play in there somewhere.

Mike Gruen:
I mean, I think there's a lot of companies that are starting that seem to claim that they have a solution. They have the silver bullet. But I don't remember who I quote on this, but there's no such thing as a silver bullet, there's just lead bullets.

Jonathan Meyers:
There is no silver bullet.

Mike Gruen:
Right, there's just lead bullets. So we just need enough lead bullets.

Joseph Carson:
I mean, in some instances, there is certain solutions or products that will actually prevent it. This one however, because of the multiple techniques that are used, if you go to the Mitre framework, and you actually plug this into the Mitre framework, it's basically the whole framework is red. They used every technique in the book here. So an organization trying to really prevent against that, you would have to have almost every solution available in the world today, and that value-wise and costly-wise, that's just not efficient and effective for business. So same to yourself, Jonathan. Since you're over at infrastructure, what things would you have been looking into, in addition to what Terence had mentioned?

Jonathan Meyers:
Yeah, so I think the interesting thing that's starting, there are a couple of companies that are starting to come up and you start to get these reputation scores based on things. And I don't think there's that many vendors that do it on domain names and traffic and things like that, where you start to get this basically raising the flag on first time domain's been seen, and things like that for an actual human to pick up. That's still I think super noisy. If you can start to tweak a lot of that type of things, and then I think the other thing goes back to maybe I'm just old school and weird, and I come from the government side of the house. But everything is segregated in networks, right?

Jonathan Meyers:
It's very different networks running very different rule engines, and so you have to be in say, for production instances and stuff like that, right? I don't know how we got away from this at some point, but outbound traffic should not be allowed. You're putting in very specific names of what outbound traffic should be going, outside of the handshake that occurs back to the client, and so I think people forget about that now because they're like, "Oh, well, we have a firewall," or "We have some next-gen thing," and it's like, "Well, cool." But outbound traffic can still go talk to whoever it wants, and I think we have gotten away from that and then segregating out your different networks.

Jonathan Meyers:
If you have source code on a network, that's a very different network than your office, where you're just sitting in there kind of doing normal browsing of the webs and things like that. And I think you can start to kind of crank up these rule engines on existing software to kind of be much more high alert when you start to see these things. I think logging and stuff like that's finally caught up to a reasonable space where you can actually log every request even if it's blocked and instead of just... like I guess we used to do in the old days, was just drop it. You would just drop it if it was blocked, and you wouldn't have any insight to how many times you were blocking things, outside of a little counter that would just tick up. You wouldn't see exactly what was blocked.

Jonathan Meyers:
But now we have this ability to kind of analyze that stuff, and I think the next step is... This sounds scary, I guess. But you got to open source, well no not open source that, but those logs have to feed into a larger threat landscape. And I think it has to be almost vendor-agnostic. We can't have these silos of you're subscribed to this threat stream, and that's where you contribute back your logs. And so only that threat stream's getting better and better. But they refuse to collaborate with another smaller company that's got a threat stream. And I think we're missing all of that, because the thing that strikes me the most about this attack was the patience. I don't know if I could have ever had that patience, because if you think about standard turnover for cyber employees and companies and things like that, this starts to outgrow that.

Jonathan Meyers:
You're looking two to three years, maybe at smaller startups and things like that. And if this thing's been sitting there for three, four years, you're now losing that every time an employee changes. And I think we need to be as a community kind of better at sharing that type of stuff, so it's kind of available to everybody without giving up IP and stuff like that. But I think there's enough ways to scrub logs and things like that, that you can start to kind of buy into this open community that shares threat streams and intels and things like that. I think that would be an interesting one of those new open source like, "We're going to sell you services on top," where it's like, "Our open source is we're going to host this logging platform that does all the analysis and stuff, and then we'll sell you services on top of that."

Mike Gruen:
Isn't that what the US government... I mean, at some point, the FBI or whomever has tried to do for US companies is, "Hey, if we can sort of collaborate, you can trust us. We're the government."

Jonathan Meyers:
But they said that was one of the reasons this was such a blind eye, is because the NSA is not allowed to spy on internal US operations. And this was an internal US operation, and so we lost that entire... Our entire security world has been set up to watch stuff leaving the country, as opposed to seeing the inner workings. Yeah.

Joseph Carson:
Yeah. And to your point, Jonathan, I think one of the things, a lot of the government is outsourced. And that's the other side of it, is-

Jonathan Meyers:
All the government.

Joseph Carson:
... it's not just basically government entities and agencies. There's so much third parties and outsourced components that the government is actually relying very much on the private industry right now. And I like your concept in theory, not only Zero Trust inbound, but also outbound.

Mike Gruen:
As Jonathan was talking, it struck me as like yeah, it's ironic that we've gotten away from this given the fact that it's so much easier to automate everything. API-driven everything is the way things are going. Everything is configurable, infrastructure is code.

Jonathan Meyers:
Yeah, look at Little Snitch. Little Snitch, do you want to allow this? Yeah. Why don't we have that for production infrastructure? Like, "Oh, this thing requested to go out, do you want to allow it?" Because Joe on his computer terminal wants to hit PGA.com, right? Oh, okay. That's very easy.

Mike Gruen:
You're right. Yeah.

Joseph Carson:
But you can do reputational-based analysis on that. While there's no one place you're going to, you can go and get threat intelligence and say, "Is this something that we know about?" Because what was very unique, we go back into the C2, it was actually unique for every customer, every piece of malware actually created that unique basically subdomain registration so it was unique for all of those, so it wasn't this common thing. But I think to your point...

Jonathan Meyers:
But if you hashed it, I think you could have done some math to have been like-

Jonathan Meyers:
... "Oh, there's something that's generating this that's consistent." And I think the only way you could have run that algorithm to learn that would have been if you had all of these names.

Joseph Carson:
And visibility of all of them, yeah.

Mike Gruen:
Yeah, it's interesting. It's a similar problem. So we depend on a vendor, I'm not going to name names, but we depend on a vendor to do some things and they run into that same problem, where they only can see what they can see. They can only see traffic that they have control over, and they're not plugged into a broader thing. So when there was a DDoS, their ability to recognize that is very limited. But meanwhile, all of those same IPs that are involved in that attack are also attacking other things. And if they were just plugged into a larger player, who could see more of the traffic across all of the internet, it's clear that those are bad IPs. So they'd be able to respond faster, and I think that that's where we need to get to, is a more shared understanding of these domains, these IPs, and reputations.

Jonathan Meyers:
I think an interesting thing is where are the ISPs and the major tier internet operators in all of this? Where does their responsibility start to fall?

Joseph Carson:
And accountability. Accountability.

Jonathan Meyers:
Yeah. Because they see all the traffic, right? There's the threat intel you need to plug into. I just need to plug into AT&T's NOC, and it'd probably be crazy how many things I'd find and predict and be able to stop. And it's weird that AT&T doesn't sell this, right?

Mike Gruen:
Or maybe they do but we're just not...

Joseph Carson:
We're not their intended customer. We don't have a special court that allows us access to it. But Jonathan, I want to pick up on some of the things you've said. I think definitely to your point is that this needs to be... and also Terence mentioned just defense and depth, and micro-segmentation or caution error, you shouldn't have this open, basically all-connected network, which I see all too common in many companies, is that basically, everything's connected to the same network. There's no isolation. You get a domain administrator account, and it has access to everything. And go back to I learned my lesson 20 years ago, working at a data center, that I should not have an account that has unfederated access to everything across the network. You should have an account that allows you real-time access just in time for the target system and it should be segmented.

Joseph Carson:
This gets into even the kind of areas that should build repositories, even be internet connected, should it even be possible to communicate to the internet? Should they be isolated, segmented? And it gets into the big question about even using things like git repositories and shared resources, how do we deal with that going forward? We can mention about the segmentation, and definitely how do we do the defense and depth and Terence mentioned? But for me, it just in some regards does seem to be overly complex, and it would great to find simplifications for that.

Terence Jackson:
It will, if I can add to that a little bit. So Jonathan, great suggestion. About a year ago, that's exactly what we did. Servers don't touch the internet anymore, because we had seen interesting behaviors from developers, no knock on developers, but if you're developing all day, and you're on a server and you're accessing multiple resources, are you going to come back to your local environment to go to PGA.com or if you're on the server, "Hey, on Chrome or Internet Explorer or whatever's here, I'm just going to watch in here." And back to Risk, we evaluated is there a need for the servers to actually connect to the internet? And probably I would say 95% was like, "No, they don't."

Terence Jackson:
We pulled down updates to a centralized location and distribute them within the network. So, that actually reduced our attack surface (and the noise that we were getting in our SIM) greatly. It didn't make a lot of people happy, but they adapted... that's a topic for another podcast. But it just really made everybody just look back and say, "Oh, well, I guess servers, we really don't need to browse the internet from a production server, and why have we not done this before?" But just to that point, yeah. It doesn't go anywhere. You can try, but it's like, nope. But we log the request, because sometimes, that's an area to re-educate someone on, "Hey, we saw this come in, you're not in trouble, but..."

Mike Gruen:
This is the reason we don't do that.

Terence Jackson:
Exactly.

Mike Gruen:
Yeah, yeah, yeah.

Joseph Carson:
How many times you log into server and you're going, "Why is Chrome sitting here? Why is Adobe PDF Reader on here?" Someone installed Microsoft Office on...

Mike Gruen:
Well, it's funny, I mean, just going back to developers find a way ... I was in a place where my local machine was more locked down than the servers. So you better believe I was like, "Oh, I need to read an article on such and such that I'm trying to work on, and I can't get there from my local machine, because my local network is more locked down. I'll just go to the server and go out from there."

Joseph Carson:
And then you find you're looking at... on the server, and it's like, "Oh, the documentation says a PDF. Hmm, okay. Let me install Adobe," and then you're find out, "Oh, it comes down, it's zipped."

Joseph Carson:
So, all of the sudden, you find your servers get bloated with all this additional software that you don't really need on there. And now you have more software you need to patch as well, so. Absolutely, the more you lock them down, the more you harden them. And I'm really a big fan of getting into the principle of Least Privilege, is that when I log on... One of the things in the data center that I kind of learned this process, we did it in separation of duties. And I realize that I should have separation of duties within my credentials as well.

Joseph Carson:
I should not have one credential that logs into every server that has the same amount of rights, and I should always start with zero and build up to what I need to be able to do. And I should not be under user context. It should always be on your basically delegated change control process. So for me, we need to get into really locking things down and preventing communication.

Mike Gruen:
And I think temporal is an important part of that. Putting a temporal time, the idea of, "I have this role for a period of time," it's not me going into an admin account. It's me being provisioned in an admin role for some period of time. And for really high risk things, maybe it is a two-key system. So where Jonathan and I worked at Red Owl, where for a short period of time before Jonathan joined, we were actually more in a legal use case thing. So we had data that was under literal lock and key, in a vault. And the way we handled that was the disc itself was encrypted, and only a handful of people had access to the encryption key. And then there was only a handful of other people that knew the code to unlock the vault.

Joseph Carson:
So I've had the same. It happens in the gambling industry, as well. When you get those games machines, in order to provide an update on that, you've got the one key that unlocks it and then you've got a second key, which is changed. So you've got that separation of duties. No one person can make the change. Same in ATMs, have that same process, is that when you go to do... That's why even when money's been transferred, that is all done in teams, is that no one single person has the keys to the kingdom.

Mike Gruen:
I think we can do similar things, and especially with automation with regard to requesting-

Joseph Carson:
A build process.

Mike Gruen:
... hey, Jonathan can see me requesting administrative access and be like, "Yeah, sure. I'm going to approve that. I know it's Mike," or whatever. We were just Slacking about it in some side channel or whatever it is. And so, we could come up with a way where we can sort of give privilege in a way temporally and not really in a way that makes it really difficult to get stuff done. At 2:00 AM, I'm sure it's a different scenario.

Jonathan Meyers:
Let me throw a wrench in that. What if your security team is one person?

Mike Gruen:
Well, right. I mean, that's a problem.

Jonathan Meyers:
Or two people. Right? Like-

Joseph Carson:
You get the cleaner to join you.

Mike Gruen:
Right.

Jonathan Meyers:
Yeah. It's like, if you're the only security guy at say, a smaller company and you've got like four devs, security guy, and then 12 other employees, does the CEO have to improve it? And then-

Mike Gruen:
Or the fact is you can have those other devs, you can get into a role where you can delegate off the responsibility of confirming that Jonathan is Jonathan to almost anyone else in the company. Anyone who you can reasonably trust with that, right.

Joseph Carson:
You're getting into non-repudiation in that process, is that you have to make sure that your audit trail cannot be tampered with, so that you become truly accountable and that somebody else can provide auditability into your actions. So if you are a single source, that you need to do all of it a one-person operation, then you need to make sure that you're auditability is accountable, that you cannot hide your audit trail. So one thing, I'd like to step back a bit, and we can kind of dive back into... because I don't think we're going to solve the issue. I think we can just bring the problems and challenges...

Mike Gruen:
Wait, what are we doing here if we're not solving it?

Joseph Carson:
I think that's something that's outside of the podcast, so that we can solve-

Mike Gruen:
No doubt.

Joseph Carson:
... bring world peace and order. But at least hopefully, it is in the right direction. And hopefully, this is a wake-up call for the industry. I just want to bring it back to some of the latest revelations that just came out this week. And even what we recommend even for those victims right now, what things they can do as well in order to maybe let's say, respond to this. So the first thing, we've heard Malwarebytes becoming the next victim that came out.

Joseph Carson:
We saw that Symantec released the news about their investigation for Teardrop and Raindrop, pieces of malware that they found in their customers. And then even today, my Twitter feed has been inundated with the latest news about now SolarLeaks, about code up for sale. Which is a bit of a surprise, because then this kind of to me is, it's likely a nation-backed group. Not maybe necessarily a nation state actually direct operators, that allows me to kind of maybe make that...

Joseph Carson:
Because there is groups out there that some nation states will ask them to do certain activities for them or campaigns, and they turn a blind eye to their profiteering and financial criminal activities to do, so forth. So based on the revelations just coming out today, Terence, Jonathan, and Mike, I'd just like to get your thoughts. Because it changes the motivation side of things for me, and it kind of raises to me that there's probably multiple groups involved, and this isn't a single group. It kind of raises that into there's potentially, we're dealing with multiple groups, maybe infrastructure, maybe malware developers, the actual execution payload campaign portion. I'm starting to see that there's potential of multiple actors here working in tandem. Any thoughts based on the latest news around what we're seeing today?

Jonathan Meyers:
So on the Malwarebytes, from what I understand, it's an old Office365 package that just happened to get hit. So I think that hopefully is more just kind of collateral... It just happened, right? They said they didn't touch any production servers. I know the CEO was immediately on Reddit as soon as that got posted. He was there, first comment was like, "Hey, I'm here to answer questions."

Jonathan Meyers:
So hopefully, they had some segmentation in their stuff, and it was kind of just this random app in the Microsoft ecosystem. But I think it's interesting, how did that app in the ecosystem, if that came out of their app store and things like that, that's very, very interesting on how it was detected there. Nobody else caught it, and it's running and it's basically probably unfettered access to their email, is what I'm assuming if that how it works. And I think it's going to start causing a lot of people to reevaluate that kind of stuff. I think Google's made some strides in that sort of thing.

Jonathan Meyers:
Their new security heightened thing, and I forget the name they branded as, but it basically blocks all third party apps from accessing your Google data outside of the core Google system things. I think that's what the Biden security team used this year. They used a physical key, and they basically locked it down so you couldn't connect it to any third party system to snip data. And so I think hopefully, Microsoft's reevaluating these whole marketplace apps type situation, because I don't think you could feel comfortable allowing the customer to make the assumption, "Yes, I want to allow this access to all my 365 data..."

Joseph Carson:
It's the wild, wild West of apps.

Jonathan Meyers:
I think that's very difficult. Yeah.

Joseph Carson:
I like the Apple approach, in regard. I think Apple could be better but I like their approach, because they have a fully federated side of things. When you get into the Google and the marketplace and even the Microsoft app store right now, it's a Wild West of plug-ins, even browsers. I get so scared of putting a browser extension in place. I have to spend hours of research watching and monitoring what it's doing before I even get to trust it.

Joseph Carson:
So Terence, I'd like to get your thoughts on that side of things as well.

Terence Jackson:
Yeah, and kind of piggy-backing on what Jonathan said, another layer of defense is we've blocked certain apps. Just for that point. Everybody's tried to make it a seamless end user experience for adding apps, but again, your end users aren't going to read what they're granting access to in your environment. App that I'm using to draw diagrams need full read access to my email inbox? Probably not. So-

Mike Gruen:
But I need this app, and I need it now.

Terence Jackson:
Exactly. That's the mindset. But you have to put some guard rails and control around that. But the SolarLeaks thing, I agree with you Joe, that it appears to be a group behind a nation state. But FireEye released a report yesterday and a new tool set to scan your AD environment, so if I had a recommendation for folks, use the tools that have been made available, whether it was through CISA or FireEye to know your exposure.

Terence Jackson:
To know if you have any of these underlying issues or configurations in your environment, and plug the gap as fast as you can. Use micro-segmentation, least privilege. At least make it difficult for them, because they may move on to another target. And I know that sounds bad, but it's like the analogy of a house with an alarm sitting next to the house that doesn't have an alarm. If I have a sign in the yard, the guy might be like, "Okay, I'm going over here to the house that doesn't have a sign in the yard, because I don't think they have an alarm." So...

Joseph Carson:
I think...

Mike Gruen:
That's why I stole my neighbor's sign.

Joseph Carson:
All your neighbor's signs are in your yard.

Mike Gruen:
I mean, I think back to that app thing Terence, that you were talking about, I think as CISO, my role is not to say no. It's to find a way to yes. And I'm having this conversation actively with someone who wants to use this third party software. It seems like it's going to really help our sales team, but it's requesting a level of access to our G suite that we just don't give to anything, let alone something that seems pretty minor. We can get along without.

Mike Gruen:
So the discussion isn't, "No, we can't do that." The discussion is, "All right, what's this tool trying to solve? What problem are we trying to solve, and can we find one that meets your needs that requires the right level of access, and doesn't require this like, "Oh, just give us the keys to castle, and trust us" level of access. I don't care how much I trust you, the fact is, you don't need this access. Why should I give it to you?

Terence Jackson:
Right. But to that point, that's a valid use case. But when random trial sites, somebody's just trying to demo something or even some of the news sites nowadays, they'll give you the option to sign in with your Microsoft account, or sign in with Google account. And some of the access that these websites want to your email is just... There's no valid business justification for...

Mike Gruen:
Yeah, I think there's a big difference between authenticate, to sign in with versus, "Oh, and then also give us access too."

Joseph Carson:
And the problem, Terence to your point, the problem with that is it's not managed. It's unfederated access, even after you finish the trial. You've actually given that application basically authentication, the OAuth in the background where it allows you to communicate, even after you finish the trial. So you're even exposing yourself to that data sharing long after you probably even remember. Who goes in and manages those? Who actually goes in and disables them after trials and so forth? So this gets a major challenge, and really for me, it's a big concern.

Jonathan Meyers:
It's an easy fix if they start putting time limits on the OAuth, token, right?

Joseph Carson:
Yes, that's-

Jonathan Meyers:
Google can flip that switch tomorrow, right?

Joseph Carson:
Yeah, I would love for that happen.

Jonathan Meyers:
Like 15 minutes, this is what you get. And then it's like, "Cool, I have to reevaluate." Kind of like Apple does with the location services, right? On your iPhone when it first asks, it just says, "Oh, yeah. Let them have access to the app right now."

Joseph Carson:
15 days, 30 days.

Jonathan Meyers:
Yeah, but then it waits...

Joseph Carson:
No longer than a year. Yeah.

Jonathan Meyers:
Yeah, and then it waits a couple times before you come back in, and then it's like, "Oh, do you want to continue allowing this? Or should we block it?" And I think that kind of approach where it comes back later and asks is an interesting play.

Joseph Carson:
Yeah, absolutely. So one thing for those right now, the victims out there, all these companies, not even just the customers of SolarWinds, but even the customers of those customers is now potentially, one thing that I got used to years ago was in this process of every now and again changing your locks to the door. And for me, is this a time where many companies should really consider going in and migrating to new domain accounts, taking their domain administrator account, disabling them and moving to new ones. Rotating credentials, is this the time?

Joseph Carson:
I mean, I think this is a pivot time in the industry where we really need to make it difficult, and not make it easy. And this means that sometimes, we have to look at renewing some of those tickets, making sure that all accounts have been logged out of systems, maybe even reboots of systems. And not necessarily changing user accounts, but those domain credentials and domain administrators that are shared, maybe this is a time where we really need to go in and proactively move to new accounts, and then monitor those accounts for potential malicious activity, so.

Jonathan Meyers:
Yeah. I think it's kind of what the government has to do, right? There's a lot of reports that are like, "The only way the government's going to know this stuff's out of their network is to completely rebuild their networks." And that sounds great from sitting back, watching this happen. But that guy, that group's life for the next three, four years, done. There's no new initiatives, there's no new anything. He's pulling, wiping everything. And I think that's going to cause some emotional problems for a lot of those people on that team.

Joseph Carson:
Absolutely. The instant response for a domain administrator account being compromised is to rebuild your active directory. You cannot continue with your existing compromised systems. It's a rebuild. And we can go through all the different scenarios, but all of those different companies that have had those SAML forgeries, that have had their certificates being compromised, that has their accounts being compromised, the realistic approach, and I've been helping a company respond to a ransomware case over the holidays. And it's a rebuild. There's no taking what you have and continuing, and this is where automation is crucial. Maybe you've got automation, great, because you can do that much quicker. But yeah, right now, we're probably in a moment in time where many companies will have to rebuild from scratch.

Mike Gruen:
Well, I think that's a great opportunity to add this automation in now, right? In the past, there's always this resistance to change or doing things. But if you're going to be doing it anyway, now's the right time. So go ahead and embrace the new ways of doing things, and do more automation and figure out ways to do that.

Jonathan Meyers:
Yeah, and I think the biggest thing that we overlook when we talk about the automation part, it remembers everything. You're not going to remember every security setting you set two years ago, and so by putting it in code and automating it, it's like, "Oh, cool. It's automatically applied, I don't have to relearn these hard lessons that I forgot." Especially when it comes to security, because something two years ago, I probably don't remember.

Jonathan Meyers:
I don't remember why I set that setting, and I wouldn't have remembered if I manually built it this time. And so-

Mike Gruen:
And the cool thing is you can even leave a comment as to why you did it.

Joseph Carson:
Right.

Jonathan Meyers:
I do that all the time.

Joseph Carson:
Post-It notes everywhere.

Jonathan Meyers:
Yeah, because I'll put in CVE numbers, right? If it doesn't make sense why I set a setting a specific way, I'll throw a CVE number in that comment so that if I'm reading it later, it's for me.

Joseph Carson:
Yeah, that's the best way. So let's close it up and I mean, because we might be talking about this in two weeks. We might be talking about this in two years. This is going to be going on for a long time. For me, this is a wake-up call, I think. It's a big wake-up call for the industry. And I think really, Brad Smith's comments at the CES Trade Show, I think that's the pivotal realization that we really need to see. This is an attack on the entire industry.

Joseph Carson:
And for me, I think my lessons I've got from this is that I very seldom use the term sophisticated. I can analyze lots of malware, I look at lots of incidents throughout my career, and this is up there probably in the top five I've had to deal with. This is significant, the patience, the skillset that was involved. For me, this really shows that cyber criminals and nation states are taking their game to a much higher level. I'd love to get all of your thoughts. What is the realization? What's your overall takeaway from this? What have you learned from this recent incident? So Jonathan, let's kick it off with you.

Jonathan Meyers:
Yeah. So I think the biggest thing that's going to come out of this is the thing that everybody's been just dragging their feet on the last couple years, is that reputation around your whole dependency management and supply chain, if it's that kind of software and things like that. I think it's going to start to get those things start coming, so I wouldn't be surprised if we see a bunch of companies now trying to do that kind of reputation based on... It's difficult with a SolarWinds product, because it's that. But hopefully SolarWinds uses it, if they're using open source and things like that. And they can start to kind of score and say, "Hey, this package that you have in your thing had one brand new committer this month with one change that he threw in here, and threw in the 30 other repositories with a brand new GitHub account."

Jonathan Meyers:
I think those type of things just brings more risk scoring, and enables you to tweak these models and make them that much more powerful. Because I think at the end of the day, AI and ML, the best thing that's going to come out of that for the next I think 20 years is it's just eliminating a lot of the noise. You still need a human sitting there making these calls, but eliminating that noise frees you up to do other spelunking in your network and finding these unique things that are happening. And I think we're going to continue on that with AI and ML for at least the next 20 years, until they get super smart. So, yeah.

Joseph Carson:
It's just an intelligence, yeah, or augmented intelligence. Terence, what have you learned from this? What will you change going forward?

Terence Jackson:
An increased focus on risk management, particularly third parties and your supply chain. But I think this allows us to have better conversations with those critical vendors and suppliers. Looking at your log aggregation strategies and micro-segmentation, you can't detect what you don't know exists. So it's back to the whole shadow IT component sometimes, of making sure you have up to date asset and software inventories in your environment. And really, training. At the end of the day, a human detected this. And like Jonathan said, we're just going to need an increased focus of eyes on glass. But alert fatigue, I've talked about this before, is real. It has to be manageable, so I think the technology will evolve to the point where we're not boiling the ocean and we will get a manageable amount of intel to act upon.

Terence Jackson:
A lot of it's just holding on for the ride, but that defense and depth approach, but this is all an exercise in risk management at the end of the day. So just making sure that your capabilities are evolving and not stagnant and not static, and yeah, you have to be able, we've said it a couple times, pivot. You have to be able to pivot on a dime often times. Because our environments and our worlds change literally every day. One day, it was like we're evaluating cyber security now. Was this before or after SolarWinds?

Mike Gruen:
I mean, I feel like it's-

Terence Jackson:
Before SolarWinds or after SolarWinds?

Mike Gruen:
I mean, I feel as if like aliens landed. All of the sudden, our whole world view would change if we learned that aliens actually were visiting the planet.

Joseph Carson:
Are they not already here?

Mike Gruen:
I mean, I was nervous about making the analogy.

Joseph Carson:
So Mike, what have you learned from even today's discussion, or even from what you've been reading in the past week or so?

Mike Gruen:
Yeah. I mean, I think so obviously, everything that Jonathan and Terence said, and you Joe, all applies. I think there's all this stuff that we can do but one of the things, and Terence touched on it a little bit. But I think this gives us an opportunity to really talk to the rest of the organization and use it as examples of why we do what we do, or to use that one person who's like, "Hey..." at FireEye, who was like, "Hey, there's this thing that's happening. I feel like this is weird that it's asking about this MFA. Why is it..." whatever. Let's celebrate that person. Let's use that as an example of, "Hey, somebody brought this to the forefront.

Mike Gruen:
Somebody reported this. That's the right thing to do," and use those positive examples that come out of this to really reinforce what we want out of... Security in depth includes everybody at the company, and this is what we're talking about. And this is a great example of how this works to prevent further problems.

Mike Gruen:
I think that's one of them. I think I'm really interested to see the attribution and how this all goes. Was it a nation state? I think there's no question that it has to be nation state-backed. How was that done, and why? I just find fascinating. It's not going to change my day to day, but at the same time, Joe you were saying, because of the profiteering aspect, it could possibly be this company that this nation state knows about but doesn't necessarily do anything or maybe they're supported. But then I think about even in the US, there's laws that allow police departments to profit off of pulling people over and just confiscating their stuff, and then selling it at auction.

Mike Gruen:
So it's completely conceivable that that organization is 100% a government organization that funds itself in part through whatever tax dollars or whatever budget they get from the government, but also from whatever property they're able to seize and resell. If I was running a government agency, I would have no problem with like, "Hey, yeah, if you guys want to profiteer off of this in this state," if I'm that type of person who's already looking to run that type of an agency, I don't see why I would draw the line at profiteering off of what I found. I'm not saying I'm that person, I'm just putting myself in that role. I'm not a terrible person. I mean, I am, but...

Joseph Carson:
I think for me, the news today, what it does is it pivots me from this not being, let's say, an espionage attack. It changes it to being criminal, that's the separation I have, is that when you bring a monetary financial fraud and profit out of it, that's what changes it from basically an intelligence gathering, espionage-type of activity to profiteering, which is a criminal activity.

Joseph Carson:
And I think then, that's where we get into the criminality side of things and holding countries against criminal intentions. So this is the difference for me, is where that crossover's starting to occur, is to start getting into that criminal side of things. So it's been awesome having you all in the panel and on the show today. This is great, and I think it's a timely discussion. I think this is one that will definitely change our industry going forward.

Joseph Carson:
And to listen to your views, and I think our audience is definitely going to get a lot of value. I'm really excited to even share with our audience today is that we've literally hit over 10,000 listens of our episodes. We launched this back in May of last year, and we continue to have the episode every two weeks. So if you're interested in listening to more, we've had some awesome guests to date and we've got some really exciting ones still coming forward. So it's due to be released. So we're excited about the show. We're excited to have our amazing audience listen to our views and sometimes ramblings, because we can go on for hours.

Joseph Carson:
It could be a complete series of one show, we just cut it into different episodes. But I'm really glad the value that our audience is getting from our special guests that we have on. And definitely subscribe to the show, get the updates. Go back and listen to older episodes. We've had some fantastic guests on like Josh Lospinoso and we had Jessikka Aro and we've had great guests. The government's been on. So if you're interested in going back, listen to our older episodes, and we'll make sure to keep you up to date on the latest news. And many thanks for listening in. So, awesome, Mike, Terence, Jonathan, it's always great to have you on. And hopefully, this won't be the last time you're on. We'll always try and grab you back on for later episodes. So out there, stay safe, listen to us, subscribe to the show. Joe Carson from 401 Access Denied. Mike, any last words?

Mike Gruen:
Nope, that's it. Great show.

Joseph Carson:
Okay. Take it away. Thank you, everyone and talk to you soon. Take care.