Finally, some culture on the program. Hahaha. Smashing Security, episode 472. AI gets hacked, and BitLocker gets bypassed. With Graham Cluley and special guest Paul Ducklin.

Hello, hello, and welcome to Smashing Security episode 472. My name's Graham Cluley.

Well, it's fabulous to have you back on the show yet again. Of course, both of us, we've been at this a long time, haven't we?

I think over 60 years combined, maybe, in cybersecurity. Would that be right?

Well, before we kick off, let's thank this week's wonderful sponsors: ProtonPass, CoreView, and Vanta. We'll be hearing more about them later on in the podcast.

This week on Smashing Security, we're not going to talk about how Cisco, the world's largest food distributor, has been hit by an extortion threat from hackers, the second one in just a few weeks.

You'll hear no discussion of how a UK police officer is being investigated for allegedly using AI to fabricate evidence.

And we won't even mention how someone used Maine's official data breach portal to file completely fake data breaches. So, Duck, what are you going to be talking about this week?

And I'm going to be talking about how your AI tools can be hijacked to leak passwords without a single phishing email or malware involved in the process.

Plus, don't miss our featured interview with Son Nguyen Kim of ProtonPass about the hidden security risks of AI agents and why connecting them to your email or calendar without a second thought could be handing attackers the keys to your business.

All this and much more coming up on this episode of Smashing Security. This episode is sponsored by ProtonPass.

That's pretty much it. All of the above. And every one of them is a breach waiting to happen. ProtonPass is built to fix exactly that.

Letting teams store and share credentials securely with end-to-end encryption baked into every feature.

It's open source and fully auditable. It runs on Swiss infrastructure, so your data sits outside US jurisdiction, and it's backed by a nonprofit.

No venture capitalists, no pressure to chase a quick exit.

Which is the bit I like. You know, it's built to serve you, not investors.

So it will never be pressured to cut security corners or rush towards a liquidity event that could change ownership, pricing, or priorities overnight.

It's trusted by over 100 million people, ISO 27001 certified, SOC 2 audited, and it helps you tick the boxes for NIST 2, DORA, and the UK's Cybersecurity and Resilience Bill.

Now, chums, I want to talk today about a type of attack which, like I said, doesn't require any malware, doesn't rely upon a stolen password, where there's no phishing emails, no bypass of your antivirus or a firewall or any other security tool you could have paid good money for.

It works by turning your AI coding assistant against you. Duck, where do you stand on AI coding assistants?

Graham, I tend not to stand. My choice is to sit down and to hold on to my chair very, very firmly after bolting it to the floor. Right.

I think the problem is that they're not so much assistants anymore, are they? They're replacements.

They're, hey, look something up, get some results and turn data into code and run it. What could possibly go wrong?

We're putting a lot of trust in them, aren't we? Yes. Now, a lot of people listening are probably thinking, well, look, I don't use an AI coding assistant. I'm not a developer.

Why should I care about this? Well, bear with me because I think this is a big deal and it can impact a lot more than just regular software developers.

So to understand what I'm talking about today, I need to explain 3 things. They're quite simple to understand on their own, but when they all come together, bad things can happen.

So number one, number one thing are the AI coding agents themselves.

So if anyone doesn't know, these days, if you're a software developer, there's a very good chance you are using an AI coding agent. Things like Claude Code or Cursor.

And these are helping coders by reading someone's code, browsing your file system, running commands directly on your computer, connecting to external devices and services on your behalf.

And you ask them to do something and they go and do it pretty autonomously.

And the latest update that I got this week of Visual Studio Code, which for my sins I use even when I'm not coding, because it's a nice text editor.

That now has a thing called Autopilot, which is Copilot that does things for you, enabled by default. And Microsoft proudly tells you that is a feature and not a bug.

Yeah, I can't imagine you'd be terribly happy about that being on by default. No.

So developers, well, some developers, maybe not Duck, they love these things because they can be genuinely useful.

But of course, as we've already described, they can be given enormous trust, maybe unwarranted trust, and of course, access to your data and systems, which could be risky.

So that's thing number 1. Okay, so everyone knows what an AI coding agent is. Number 2. Thing number 2 is Sentry. Now, Sentry is an error monitoring tool.

It's been part of software development for well over a decade now.

So when your software crashes or when it goes wrong, out in the world, so it's in real life, you know, not just in your coding environment, and it creates an unexpected error, Sentry will log the error so your team of software engineers can investigate later.

It's a little bit like how when a program crashes, sometimes it says, would you like to send a report to the developers with the details of what went wrong so they can do whatever it is they're going to do with it?

But it's more than just a smoke alarm, isn't it?

It's a smoke alarm that when it goes off, even if it's a false alarm, it takes a photograph of your flat and anyone who's walking around, and it takes all readings from all your smart meters and it sends them back to somebody else's head office just in case.

So it may be that Sentry is running on a web application.

So it could be a website that you visited and you went there with a funny browser or with some other programs installed as well.

One with a comedy nose and clown shoes. Absolutely. So then the message gets sent to the developers and so they can hopefully analyse what went wrong.

The way that Sentry receives these error reports from your software isn't through an email address. Instead, it's through a public web address.

So the address is embedded in a website's code, which means that anyone visiting your site can see it. And that's the way it's meant to work, right?

It's public, it's out there, it's not private. And that's always been fine because the communication is one way only.

Anyone can send errors in, but only authorised authenticated members of the development team can read them back out.

So it's not a doorway, it's not something you can go in and come out through. It's more like a letterbox.

People can drop messages through about how your software has crashed, and you can pick up those letters and think, oh well, okay, we know what we have to fix now.

And that's fine, or at least it was for years and years.

Yes, they could. Oh dear.

And obviously that'd be a nuisance if they were to do that in an automated way, particularly because you could just get a deluge of nonsense coming in all the time.

But it's not supposed to be dangerous, right?

They can't send you a report that says, "And by the way, crash your car on the way home or else." Well, no, obviously any developer reading such a message wouldn't go and crash their car on the way home, would they?

Maybe you can see where we're beginning to go here. So let's come to thing number 3. Which is the connection between your AI agent and Sentry.

So modern AI code agents can plug into tools like Sentry. They can read back all the unresolved errors in your software and help you fix them.

Pretty helpful if you're getting a deluge of feedback, isn't it? And this all happens through something called the MCP, the Model Context Protocol.

It's a nerdy term I'm not going to mention again, but basically means there's a standard that lets AI agents connect to external services.

And when your AI agent reads data back from one of those services, it treats it as trusted and authoritative. After all, it came from your own Sentry account.

So why would it be suspicious of data from your own error monitoring tool?

And I think, Duck, you already had the idea of this message being sent in saying something unpleasant or saying something nasty, a booby-trapped bug report, because that's what we're dealing with.

It turns out anyone can post a fake error through your Sentry account's letterbox.

No password required, no authentication, and you can make that fake error report say whatever you want.

As if you can insult a developer, criticise their curly brackets, because the AI isn't going to get insulted. This is basically telling the AI, go out and do something terrible.

Is that right?

Not Telnet, no. Not Telstar, not Tenant, Tenet.

And they described how they'd crafted fake bug reports that looked entirely legitimate, so the right formatting and structure that would fool anyone who didn't look carefully.

But hidden inside each one was a fake instruction formatted to look like official guidance on how to handle a bug report from Sentry itself.

Oh, as if Sentry was helpfully telling the AI how to fix the problem.

So all a bad guy would have to do is wait, wait for a developer to open their AI coding assistant and say, "Hey, can you look at our unresolved Sentry errors and help me fix them?" Oh, so if it doesn't actually stumble upon your error report by itself, you can just call up the help desk and kind of help the whole thing along.

Oh, absolutely.

So the agent connects to Sentry, reads back the errors, including the planted fake one, and it cannot tell the difference between a real error generated by your software and a fake one planted by an attacker.

They look identical. And so the fake instruction in the error report looks exactly like legitimate guidance on how to fix a bug.

And so the AI agent does what agents are supposed to do. It follows the instructions, runs the command that the instructions have told it to, oh, this is how you fix the bug.

And it goes, oh, thank you very much. I'll go and do that because I trust you.

So it will then run it on the developer's machine with the developer's privileges while the developer sits there thinking their AI has just helpfully investigated a bug and is fixing it.

So this then means that the code planted effectively by the bad guys now has the developer's privileges on their own machine.

They can reach everything the developer has access to, including AWS keys and GitHub tokens and database passwords and all of it.

And that can be gathered up and sent back to the attackers.

—authorise their AI assistant look for the errors and the AI connected to Sentry via an established integration that was authorised.

And the AI ran a tool that it believed had been authorised to run.

So good luck with your traditional security tools flagging anything if you've plugged AI deep inside your organisation, there's this chance if you're acting like a regular developer right now in 2026, that something like this could happen to you.

So I think this is not that great.

But no one should ever fall for running an AI and allowing it access. No one should really be running a Agentic AI, should they?

I mean, to be honest, unless you absolutely have the tightest guardrails imaginable upon it.

Unless you've actually got it on reins like a 3-year-old at a theme park, you want to be able to yank it back, say, what the bloody hell are you doing there?

Yeah, it just beggars belief, right?

This sounds as fatuous and as silly as an attack basis as those things you see in older bank heist movies where they take a Polaroid photo and hold it up in front of a CCTV camera and everybody falls for it while they wander around the bank for 20 minutes blowing things up.

I mean, it sounds bat crazy to me.

Yeah, but I think in the rush to integrate AI into organisations, I'm slightly sympathetic with developers because developers obviously are terrified of losing their jobs because AI is a quick coder.

It may not always be the best quality, but it's good enough and it's a hell of a lot cheaper.

So the people who do still have coding jobs are going to be thinking, how can I harness AI to make myself more efficient and produce more code?

Because I'm competing with machines now.

Well, we're already hearing stories of companies that at least claim that they measure developer productivity by how many AI tokens they consume.

Which is just like the old 1970s IBM metric — basically, if you didn't write enough lines of code in a day, then you were deemed to be a rubbish programmer, which drove the behaviour that you just churned out code as fast as you could and didn't care whether it was efficient or safe.

Which is how we got into cybersecurity problems in the first place that we're now throwing ourselves back into. So it does seem a question of throwing yourself under the bus.

So what I'm interested in is what did the security researchers at Tenet do with their discovery?

So they didn't just demonstrate it in a lab with a test account — they actually went out into the real world.

They found 2,400 organisations with exposed Sentry accounts, including some big name organisations.

And then using what they described as carefully limited self-identifying payloads that didn't actually steal anything.

They ran their attack against over 100 real organisations to prove that it worked outside a controlled environment.

So their payload did identify itself as a "tenant security scan," in quotes.

And rather than grabbing credentials, it just phoned home to confirm that the agent had executed it and checked whether certain sensitive files existed on the machine — not all of them, and not what was in them.

But they did that and it worked 85% of the time.

Like going, hey, I went looking on your system for a file called banana.dat and I found one. Like you have to have acquired unauthorised access to do that.

That seems a bit dodgy, wouldn't you say? And maybe they could have done 3, not 1,003.

Right, right. Yeah. So they say it was responsible security research. They say they were careful about what they collected.

They notified, presumably afterwards, the affected organisation — it's not like they asked permission beforehand. But they did access other companies' accounts without permission.

They did cause code to execute on developers' machines without those developers' knowledge or consent. Who knows whether that could have crashed something, or done some damage?

Or what if there hadn't been much hard disk space or it was low on memory? You know, it's like, you can't do that, can you?

Sometimes when I moan about things like this, there are people in the security community who would say, oh, come on, granddad, we don't live in that world anymore.

I feel like that still feels a bit naughty to me.

Yes, because it's not for you to decide that your code won't cause any harm.

And also, if you look at, for example, and this has been done in the US, I know it's been done in the Netherlands, that when someone has known malware on the computer that opens them up to abuse by any Thom, Dick, or Harriet anywhere in the world, sometimes law enforcement will get a court order that allows them to go in and exploit that vulnerability in a very specific way to close down the malware.

And even when they do that, the law enforcement authorities do admit, we know this could go wrong. We had to jump through hoops. We had to go to a judge. We had to get a warrant.

We had to show the code we were going to execute. We had to dot every I, cross every T. So that is very much a thing in the modern world, actually being careful.

You think they could have found one company that would agree to provide them with a test environment where it could be done safely. And that's all you need, right?

So I don't think you're being a granddad there, Graham.

I think that once you start letting those standards slip, then you can't point at a real cybercriminal or a ransomware crook and say, how dare you scramble my files and then ask me for the money.

And claim that you're a postpaid penetration tester.

So Tenet did contact Sentry about this. And Sentry responded the same day. That's obviously good.

You know, some vendors may have taken weeks and they said the problem was, quote, technically not defensible on their end.

So they basically sort of washed their hands of it and said, well, you know, nothing really we can do about that.

So I guess what they meant was because the public address has to be public, because this is the whole sort of way in which their system works.

It lives on a website and JavaScript that anyone can read. You can't verify who is sending errors to it because they want anyone to be able to send errors to it.

So what they have done, however, is they've blocked the specific payload string that Tenet used in their tests.

But of course, that was a specific payload string, and that isn't really fixing the problem. The technique still works.

So I do feel some sympathy for Sentry because I also think, well, hang on, isn't this the Agentic AI's fault? Because why is it not being a bit smarter?

Human intelligence would have been more suspicious, I suspect, than the AI would have been.

I agree with you feeling a little bit sorry for Sentry there. What are they supposed to do? They submit data, and it's up to the person who receives it to decide what to do with it.

After all, if Sentry submitted this data and then the company had an insecure storage bucket that they collected it in, so that all this data just leaked, would that be Sentry's fault or would that be the service provider's fault?

So I feel like we're talking about AI every week these days. It feels like cybersecurity has just become a whole much bigger problem because of AI.

But if an attacker can plant text somewhere that your AI agent will read, it's possible that your AI agent will act upon it, and that may not be good.

And once again, it feels like we're rushing into plugging these things in without having the proper security in place.

And maybe we're being a little bit too rash to do some of these things. Well, we've got time now to talk about one of today's sponsors, Vanta.

Joe, what keeps you up at 2 o'clock in the morning?

No, no, Joe, it's Vanta with a V. It's a trust management platform. It's not a drink full of sugar.

It automates all of that tedious manual compliance work so you can stop drowning in spreadsheets, chasing audit evidence, and filling out questionnaire after questionnaire.

Vanta continuously monitors your systems. It centralises your security data. It keeps your programme audit ready all of the time.

It also uses AI to streamline evidence collection and flag risks. It automates compliance for SOC 2, ISO 27001, HIPAA, GDPR, and more.

Well, I want to talk about something that has also been dominating the news, perhaps not quite as much as all the excitement over AI, but certainly has been all over the news.

And that is, in two words, Nightmare Eclipse.

Oh, see, I thought when you said Nightmare Eclipse, I thought that must be some new fashionable perfume, but the stench of Steve Ballmer or whoever runs Microsoft these days.

Okay, so what is Nightmare Eclipse?

Nightmare Eclipse exists as an anime avatar. Right. That's the only visual representation of this person, or for all we know, it could be a group of hackers and crackers. Right.

Basically, the backstory is they submitted a bug report to Microsoft some time ago, and they provided proof of concept code and a description and everything.

And Microsoft came back to them and said, thanks for your bug report. We don't accept bug reports unless you make a video showing it working. And until then, it's not a bug.

We don't care. You can't get a bug bounty and we're not going to look at it.

No, I don't think it's quite that bad. And you could argue that if the exploit works well enough, then maybe a 1-minute screencast video isn't that hard to make.

But Nightmare Eclipse basically threw their toys out of their cot and said, well, if you don't want to accept the bug report because there's no video, then there can't be any objection if I just publish it for everybody.

I do what's called full disclosure. I think it's a bug. Administrators might be interested in knowing it's a bug.

And there is a school of thought that says don't wait for vendors, don't do responsible disclosure, if we just always tell everybody at the same time.

The bad side of that is the crooks get hold of attacks on day zero.

But the good news is that well-informed administrators don't have to wait for vendors to come to the party, run around for weeks, wait for videos, maybe try and brush things under the carpet, etc., etc.

So Nightmare Eclipse decided that they would release this to the public, and just to grind their axe a little bit sharper, they published two other zero days at the same time, and they chose just after April's Patch Tuesday to do it for best PR purposes.

Yes. All the system administrators who've pushed out all those patches have gone, oh, I wonder if anything's going to go wrong this month.

But they've scheduled the time and their bosses have given them the budget to do it on the Wednesday and Thursday.

And they're thinking, maybe I can just relax a little bit and do something else for the next 4 weeks. And bingo, then comes this massive exposé.

And very embarrassingly, those first bugs that came out in April actually— I shouldn't laugh because it isn't funny, but it did make me smile.

The bugs exploited security holes in the very software that Microsoft sells you to keep the bad guys out, namely Microsoft Defender, which is their built-in antivirus, right?

That's right. And all its other stuff.

And in, I think, two of the attacks, to get Defender to misbehave, they needed to provoke a malware detection, which obviously is going to draw attention to the attack, except that they deliberately dropped a copy of the EICAR test string.

It is a text string and was a simple coming together of well-meaning antivirus companies at the time to fight against what some of the more maverick players of the day were doing, which was actually handing out real viruses to their customers to test that the software was installed and would generate alerts correctly.

Oh dear, what if it doesn't work? Yeah. So the idea is it is not meant to test that a product's good at detecting malware.

It's not meant to generate alerts that throw you into a panic.

It's just meant to be a simple way of triggering a file detection on a system so you can check that if you have an alerting mechanism in place, that the alerts flow correctly.

So by simply writing the EICAR file to disk, they could create an alert.

To this day, pretty much every EDR, every threat prevention software that's out there will detect it because the reasons that made it a good idea in 1990 are still a good idea today.

And in fact, the whole idea was Nightmare Eclipse did not want to infect the machine with malware.

They simply wanted to send Defender down a special code path that it only took when it was dealing with a virus attack. Right.

So this is peculiarly embarrassing for Microsoft that their security software, their gatekeeper program, turned out to be a backdoor that allowed people to do an exploit.

That's just the beginning. Because the month after, during the month of May, Nightmare Eclipse did much the same thing again.

But this time, the main exploit they produced was one called Yellow Key. That was basically a bunch of files. They were only data files.

There was no code in there, no scripts, nothing that would trigger even the most inquisitive antivirus software, you'd imagine. Looked completely innocent.

You copy those files onto a USB stick, you put that USB stick into somebody's computer, you go Shift+Restart from their lock screen, which gets recovery mode, and bingo, you bypass BitLocker full disk encryption completely if it is set up in default mode.

So this is extraordinary. So I mean, it is full disk encryption.

The whole idea about it is that if you lose your laptop, for instance, no one will be able to get in and access your data because they don't know your password, which you've used to encrypt your drive.

But you're saying with just a USB stick with this bunch of files on it. Yes. There's a way to actually bypass BitLocker so you can access what is on the disk.

What's supposed to happen is when you boot into recovery mode, a light blue screen pops up — like the blue screen of death, but it isn't.

And then you get some menus, very, very big and basic menus that you can click on with the mouse.

You can get to a thing that says, give me a command prompt, which allows me to access my C drive. And that way you can try and fix it. You can copy off files in an emergency.

Basically, you can rescue a ruined disk if you're lucky. So it's very, very useful to do this.

However, before you get to the command prompt, before you can type in C: Enter and see everybody's files on the entire disk as the local system account, you have to put in what BitLocker calls the recovery key or the numeric password, which is a 48-digit randomly chosen string.

The theory is basically nobody's going to guess it. But with the Yellow Key bypass, you just skip the menus and the drive unlocks itself automatically. No user intervention required.

Well, it sort of is and it isn't.

I think the most disastrous thing about Yellow Key perhaps is that one of the reasons companies use BitLocker on all their company laptops is not just that they want to protect their customers' data and that they want to look after their intellectual property.

Let's hope that they do.

But loosely speaking, in many countries such as the UK, if a laptop gets lost or stolen and you can show that you were using full disk encryption set up to some minimum standard, then because of the encryption and because of the password, you don't have to treat it as a data breach.

This kind of blew that away retrospectively.

Because you can imagine a crook who stole a laptop 6 months ago and they haven't got around to selling it yet and thinks, oh, I'm not going to get anything off this.

Eventually I'll just take out the hard disk, I'll put in a new one, and I'll try and sell it for 50 quid or something.

Something, can now go, hey, why don't I just put in a Yellow Key, magic key, and reboot and see if I can get some data off. Then I can sell the data.

In other words, CISOs must have been thinking, I wonder if I need to report, say, the last 6 months of laptop thefts, given that those laptops probably haven't been disposed of yet.

They might still be in circulation. And they're no longer protected, really.

Why is this even possible?

Well, I mean, it sounds like this has almost been coded into it, because you would think if the drive is encrypted in the first place, why would there ever be something which allowed you to circumvent that check at that point for that recovery key?

Well, this is something that Nightmare Eclipse themselves cottoned on to because they don't have to prove this. They just have to sow the seeds of doubt.

And they wrote in their original report words to the effect of, "Hahaha, who knows? Maybe this is a deliberate backdoor. Only Microsoft can say," like doxing.

So they don't have to prove that. They just have to say that. And then, yes, people might be thinking, yeah, like you've just asked, why would you put such a bypass?

Now, the reason this works is actually because the default mode of BitLocker, and sadly the one that is preferred by a lot of IT departments, is what's called TPM mode.

It's an admittedly controversial chip that modern laptops have inside them that can securely store things like cryptographic keys.

Keys that can only be extracted and used under special circumstances, like during the Secure Boot process.

So Windows 11, by default, strictly enforces that a laptop must have this TPM chip to store cryptographic keys, and it must have a thing called Secure Boot, which is supposed to protect these keys from being manipulated by someone who isn't an administrator.

And therefore, the way that BitLocker works in what's called TPM mode is it automatically extracts your full disk encryption password from this supposedly super secure chip during the super secure boot process and seamlessly and transparently unlocks the drive.

Now, as crazy as that sounds, if the TPM chip and the Secure Boot process work correctly, it does provide you with at least some security because you have to put the hard disk in that laptop and you have to start it up and it then only goes down a code path which is supposed to take you to the Windows login prompt.

I know that's a big if, but that's the theory.

And users and IT managers love it because you don't have to remember or enter some kind of PIN or password every time you turn off and on or lock and unlock your device like you do on a mobile phone.

The other thing that companies like about it is because that chip is in the specific laptop, it means if someone steals the laptop and takes the hard disk out and puts it on another computer, it won't unlock because that computer doesn't have the right chip.

So it ties the disk to the laptop. So it's not a useless idea. It's just, if you like, the minimum you can do to make things safe.

So there is a mode you can use for BitLocker called TPM and PIN where — right, you need to have the hard disk in the right laptop and there's a PIN, and you can even make it a long password that you have to put in right at the start when you boot up.

If you can choose that mode, if you can convince your users as an IT manager — Smashing Security.

Crypto experts have been advising people not to rely on this automatic unlock mode for years because there are just too many points at which a vulnerability could be introduced.

So that does protect against this attack, but by default a lot of laptops were exposed.

And although I'm not aware of anyone having data exfiltrated from their computers in this way, it was rather a teachable moment.

And a scary thing for sysadmins around the world, like this premise they'd been clinging on to for years, that this automatic chip-based unlock mode in Windows 11 that's supposed to protect their systems from data breaches maybe was not quite as solid as it had seemed all along.

Well, they did. I mean, Microsoft owns GitHub, so I think they just press the button, gone.

But they also published a blog article where they said full disclosure, which they call irresponsible behaviour. That's always unacceptable. Always?

Even if a vendor won't play ball, we support coordinated disclosure, as they call it, responsible disclosure.

By coordinated, they mean the vendor should get a say in the timing and the messaging in the actual response. And we think anything else is unacceptable.

Largely, the security community would agree, but A, there are exceptions, and B, there are people who say no, full disclosure is the only way because it's the only way we can have an unequivocal rule that's not flexible or where you can't favour your buddies if you want to.

Then they said, and by the way, anyone who publishes this kind of stuff is pretty much as bad as the crooks who go on and use it because they're aiding and abetting crime.

Those weren't the words they used. We're going to make sure our Digital Crimes Unit is all over this kind of thing.

Well, I was wondering that because I get the point. Nightmare Eclipse, they explicitly have an axe to grind with Microsoft.

They've used quite aggressive words about, you know, how they want to grind their bones, all this kind of stuff.

But yes, they are upset. And they are prepared to use Microsoft's customers as pawns in all of this by talking up these attacks.

So I get why Microsoft could be offended or aggrieved or think this is no good.

But in that case, surely they shouldn't just put out this generic threat, we are going to sue or do a prosecution against anybody who publishes this kind of stuff.

They could say, we think this person is behaving in a way that's unacceptable, whereas others who publish stuff on GitHub that is potentially dangerous are maybe behaving in a slightly better way.

But I absolutely agree with you. I think it's hypocritical that they closed down Nightmare Eclipse's account.

I mean, I'm not saying they shouldn't be allowed to do that if they want, because this stuff is dangerous.

But then why are malware source code, malware analysis, network sniffing tools, ransomware samples — hey, here's how you do the encryption if you want to write ransomware — why is a tool like EvilEngineX, which you may have heard of, full of stars and voted up as this fantastic tool that Microsoft seems to love to have on GitHub because it can be used by red teamers and penetration testers?

Basically, EvilEngineX in 5 minutes can clone somebody's website, make a pixel-perfect, JavaScript-perfect copy, and basically start a live phishing attack for you with the ultimate goal of stealing things like usernames, two-factor authentication codes, passwords.

Tell me that benefits users more than it benefits cybercriminals. But apparently it does.

So it did seem that Microsoft had maybe rowed the boat out a bit too far, and it seemed that they rowed it back. They published a follow-up that wasn't very explicit.

They didn't say, okay, Nightmare Eclipse is off the hook.

They just said, okay, we're kind of saying that we don't think we'll prosecute individuals who are doing actual cybersecurity research and publishing the results.

And they did apparently allow Nightmare Eclipse to create a brand new account on GitHub.

This one, the username is MSNightmare, although their display name is still Nightmare Eclipse and they've still got an anime avatar. Which seemed a nice thing for Microsoft to do.

And in response, Nightmare Eclipse has very kindly in the month of June, just after Patch Tuesday, dropped two new zero-day exploits.

Ransomware again, one of which relies on exploiting a hole in Windows Defender, and if you don't mind, also targets BitLocker. So, oh my goodness, watch this space is all I can say.

Well, listeners who are interested in this, Duck has written a series of great blog posts up on the Sophos Cyber site. We will link to them in the show notes.

We can read much more about all of this and take some of his advice there on how to perhaps protect your organisation. Now, time for a quick word from our friends at CoreView.

Joe, quick question for you. How confident are you in your Microsoft 365 security posture?

Oh, for goodness' sake, Joe, it's for our sponsor. Just play along with me, right? Picture the scene. It's Monday morning.

You've got your coffee, you're wearing your second best hoodie.

You're feeling pretty good about your Microsoft 365 setup because you checked Purview, you tightened conditional access, and frankly, you deserve a biscuit. Biscuits?

Okay, I'm in. I'll play along with you. Thank goodness for that. So, and then someone forwards you a breach report about a company that did all of that too. So how did they get hacked?

Turns out some quiet little permission that crept wider over 3 years. A policy exception that nobody had reviewed, the kind of thing that's invisible until it isn't.

And this is exactly the stuff that CoreView's free Microsoft 365 Security Posture Check tool is designed to sniff out.

It's the drift, the exceptions, the little permissions you stopped looking at because, well, you assumed they were fine. And the spoiler is that they're often not.

It's free, it runs locally on your own machine, it does not send your tenant data back to CoreView or anyone else for that matter.

And if you'd like a hand setting it up, their team will happily walk you through it.

Pick of the Week is the part of the show where everyone chooses something they like.

Could be a funny story, a book that they've read, a TV show, a movie, a record, a podcast, a website, or an app. Whatever they like. Doesn't have to be security related necessarily.

And also, if it is a tune, it's only a real song if you can whistle it, is my opinion. Oh, Graham, come on. No, it's true.

If your milkman isn't whistling, as though I have milkmen, if you can't whistle it, it doesn't exist.

My pick of the week this week is a bit security related. Inside a large warehouse in Huntsville, Alabama, the FBI has built a small American town. Inside a warehouse.

Inside a warehouse, a large warehouse. Yeah. It's got a courthouse, a hotel, a petrol station, a gas station, I suppose, an arcade, hospital, traffic lights, fully furnished houses.

It's like The Truman Show.

Well, I love it when you go to a model village and inside the model village, it has a model of the model village.

And then if you look really close, I went to one of those the other day.

Finally, some culture on the programme. Anyway, you may be asking, why has the FBI built a small town inside a warehouse? And apparently, it is their kinetic cyber range.

This is an indoor training facility, 22,000 square feet, designed to teach law enforcement how to investigate—

It's designed to teach law enforcement how to investigate and respond to real-life cyber attacks.

So, everything in this place is fully functioning, it's got systems, devices, IoT equipment, servers, all wired up, behaving exactly as they would in a real community.

They are using this to train students with real hands-on experience rather than just learning the theory in a classroom.

And apparently since February last year, it's trained nearly 1,400 students, not just FBI agents, but the US Army, local law enforcement, NASA as well.

I do remember they took a virus once up to the space station, didn't they? They managed to infect themselves. Yeah. But it went up on a USB stick.

Yes, I think so. Yes. So anybody who ever said, oh, we've got a 2-metre air gap between our secure network and our insecure network — how high up is the space station?

Is it like 400 kilometres? Bloody high up. Oh dear.

I must admit, it sounds kind of silly when you first mentioned it. I thought, oh, 2,000 square metres, that's like a massive house — surely you could just do it in a lab.

But I guess the stuff you can do here is you can have real people in the way. You can have desks full of people who are getting agitated and anxious.

You can have coffee machines that do or don't work. You can have server rooms where nobody can remember where the key got left. And are you going to smash the window?

You know, you can have crawl spaces where you have to get in there — if you want to do a disconnect, you've got to get in there and—

They've got it all here. They've got a bloody arcade with video machines. I mean, they are having a blast, the FBI.

I don't know who's paid for all of this, but apparently it's all doing excellent work. And so I will link to it in the show notes so you can check it out for yourself.

I was sceptical at first, but I just like the idea that there will be doors that are locked, there will be windows that don't open, there will be server rooms where there's not enough room for two people to go in at once.

There will be cantankerous jobsworths who won't let you into the courthouse. You know? Imagine what fun you could have.

My pick of the week is — I've had a Raspberry Pi Zero W. That's one of the old tiny little Raspberry Pis that I've had kicking around for several years.

They're quite old and now considered no good. You need to get the Pi Zero 2, which is a 64-bit ARM chip, etc., etc.

But it turns out that there are still Linux-based distros that still support it pretty much as a first-class citizen, like Alpine, for example.

And so I decided, well, it's sitting there doing nothing, it's got an SD card in it, why don't I just set it up as a little USB-powered router that I can take with me to coffee shops?

Because there are a few coffee shops that I like around Oxford that have tired old Wi-Fi equipment where either your mobile phone won't connect to it because it's just not secure enough, or you just think, you know, no, I don't think so, not going to connect my laptop directly to it.

And now I can plug my laptop via a USB cable, which acts as an Ethernet port, into my Raspberry Pi Zero.

I can connect from the Pi Zero onwards to the Wi-Fi I definitely don't trust, I can put a whole load of lockdowns in place because it's still powerful enough to do even something a little bit like Pi-hole, you know, ad blocking, could even do that.

So that's what I've been doing. So my pick of the week is not so much the Raspberry Pi Zero W, or Alpine Linux, both of which are great.

But my pick of the week is the idea that you may just have some old gadgets lying around that are not as old or as useless or quite as ready to go into landfill as you might have thought.

Oh, hear, hear to that. A great pick of the week. Well, we've got time for another guest now on the podcast, and I'm delighted to be joined by Son Nguyen Kim.

Son leads ProtonPass, Proton's privacy-first password manager for businesses. Son, welcome to Smashing Security.

Now, Son, I want to start with something I suspect a lot of our listeners are quietly guilty of, which is that small businesses everywhere are plugging AI tools into their systems.

They're connecting them to email, calendars, internal databases, all kinds of things. And mostly they're just clicking through the permission screens without reading them.

From where you sit at ProtonPass, what do you think that those companies have actually just done to themselves by doing that?

Yeah, so AI integration is very easy, is very smooth. But behind the scenes, we need to know that we are giving access to a special agent.

It's like a human but never sleeps, can act really fast, can do a lot of things on its own, and it can listen to anyone reaching out to it.

So for example, if someone can talk to the agent, they can convince the agent to do things that can actually harm our business.

And that will only get worse because usually when we accept integration, we don't really look at the permission or scope and we just approve everything, you know, to make it fast so the agent can start doing things that it needs to do.

And then we don't really have any monitoring system to know what the agent is doing, or any alert system to know that the agent is doing something that might be harmful.

So kind of the summary that I would tell everyone is it's not just a tool. You should see it as a new employee that you onboard to the company.

Right, you give them the access to the most important data of the company and you can skip the background check.

And this employee might be naive, might be tricked by bad actors into doing things that it's not supposed to do without telling you. So be super careful with that.

So there's a number of problems here. One is, as you've identified, is that the AI tool you've effectively allowed to become a privileged insider inside your company.

It's like an employee, but one that hasn't gone through the interview and check-in process, but also that they have this sort of unscoped broad access that you've granted a third-party system to them.

So they've essentially been handed a set of keys without much thought about who is actually holding them.

And one of the concerns is that stolen credentials have been a number one entry point for attackers for years, haven't they? I mean, we hear this at every security conference.

Is what you're describing just more of the same problem but dressed up in new clothes, or is this something genuinely different which is happening here?

So what's new is autonomy. Agents have autonomy and agents can act way faster than a human. An agent never sleeps. It can work 1,000 times faster than a human.

It can do a lot of things very quickly. And another thing is an agent can be convinced by a bad actor to do bad things via prompt injection, for example.

So let's say if an agent has access to some data that can be controlled by a bad actor.

Let's say the agent visits a website, and on this website there's hidden instructions that tells the agent to send all the emails in your system, forward all the emails to an email address that the hacker owns.

You're not going to see it, but behind the scenes, the hacker will gain access to all your emails. That can happen.

So I would say the mechanism to authenticate is the same, but the behaviour around it is new. It's way faster.

It can be social engineered and we don't have enough monitoring or alert system to know what's going on and to intervene when needed.

So we've got problems of speed. These AI agents, they have real velocity, don't they? We have autonomy as well.

They're acting without human approval and the access which they have is really frightening because they can access so much information.

But can you paint a picture for me of what a breach involving AI agent credentials actually looks like for a business? So something you'd actually see happening.

So one concrete example is let's say you have an agent that is connected to your email and answers customer support questions.

An email came in that actually contains a poison input, a malicious prompt injection.

It can be something like, ignore all the previous instructions and follow what I'm going to tell you.

And the hacker can then tell the agent to do things like make a purchase, send the money to another bank account, or review all the emails that the agent has access to, forward the invoice, exfiltrate customer data, anything.

And the worst is you don't know about that because you've granted access to the agent, you trust the agent to do things on behalf of you.

And because of that, there's no alert, there's nothing abnormal that you're going to see.

So basically humans are blind in this case, and maybe they're going to realise that sometimes later, but it's already too late.

So there's real danger here of your data being exfiltrated, your intellectual property maybe.

If you have something like an agent plugged into your email, there's potential for business email compromise because the agent can access your calendar and your email contacts.

So there are opportunities for financial fraud. It's a pretty sobering picture. You're describing what seems to me to be like a third-party risk, but it's faster.

And because it's AI, it's also at scale as well. But surely a forgotten service account which has sat unmonitored for months is just as dangerous as something like this.

What makes the AI agent version of this meaningfully worse?

So you're right that a forgotten service account is also very dangerous. Something that we don't pay attention to that can do things in the background without triggering any alarm.

But the thing with agents is it just makes it faster with more impact, and especially for people who never managed service accounts before.

So a lot of people who enable agents don't have the technical background to know what is actually a service account, right?

Service account is a technical word that not everyone is familiar with.

And then because right now we have kind of the FOMO going on, fear of missing out on AI agents, everyone wants to integrate AI into their workflow and they want to do that fast.

You know, they want to spin up maybe 5, 10, 50 agent integrations in weeks, in months, and then they forget about it. But the agent doesn't forget, the agent doesn't disappear.

They're still there. They still listen to instructions, maybe from you or maybe from someone else. And then because of that, you don't know that it exists.

For non-technical people, they just don't have the technical knowledge to monitor all of them or to know what's going on.

So we've talked in the past — it's not a new idea — things like least privilege and scoped access. Security teams have been preaching about them for years and years and years.

Why does it feel like they are being thrown out of the window the moment companies start deploying AI agents?

Is it that fear of missing out, do you think, or is there more than that?

It's kind of related to the FOMO in the sense that we want to do things very quickly, the quickest way possible.

So usually people will just accept the defaults, and by default the agent will ask for as many permissions as possible so it doesn't have to ask again.

So everything will work out perfectly at the beginning, so people just click allow all and then the agent will have access to everything.

The second thing is scoping is actually quite hard — people need to understand what a permission actually means, and they need to know what permissions the agent actually needs to decide which ones it should have access to.

And also related to the FOMO, people want to do that fast.

You know, I just want to have this agent working right now so I can see the benefit, so I can show to other people that I'm an AI-native person.

Yes. And there's so much pressure on employees now to get lots of work done. And it's not as though AI is necessarily making our lives better.

It can be that AI is just helping us do more during our working day, and we feel like we need to use AI to keep up with our colleagues and with our managers' demands.

And I imagine one problem is that there may be a situation where the people who are actually turning on the AI or onboarding it in a particular app may not be the IT and security team.

They may not be in the loop when business users are adopting these tools.

So there's a gap, isn't there, between what people know they should be doing and what actually happens under pressure in order to stay competitive.

So there are probably people listening right now who are thinking, I genuinely have no idea what access my AI tools have actually got.

They're probably thinking, where do we even start?

So there's no way that just sitting down and trying to remember all the agents, integrations that you have enabled.

Maybe going to all the tools that you use, email, calendar, etc., and check which agent, which integration is enabled.

And then for each agent, try to ask the three questions — what can it access? So what scope did we grant to it, read or write?

Every permission or just some permissions, and who owns it, and who's going to know when it's not behaving correctly.

And then try to find the credentials that the agent has access to. Is this via a config file? Is this via a secret manager? Is this maybe an employee's personal account?

And from that, trying to reduce the scope that the agent has and maybe talk with the person who has activated the agent and ask them why they need the agent and try to reduce the scope that they have granted.

That can take a lot of time to go through everything and talk with everyone to understand their needs and reduce the access, the scope of the agent.

But that's the first thing to do.

So the first thing to do, step one, is getting some visibility on what's happening and then what scopes those apps have been granted and then going back to the users and saying, what do you use this for?

Do you really need this? That's something which IT teams can do, hopefully.

And once you've got that picture, if things do go wrong, I guess you have to consider how quickly your company can actually cut off access to an AI agent which you've decided is risky.

What does the revocation process look like in practice for doing that?

So let's say you have a list of all the AI agents and what they have access to, and how to set them up. In theory, it's quite easy to revoke the access, right?

You can just go to the settings and remove the access from the agent. But what we don't know is what's going to be the consequences, right?

Maybe the agent is used in the sales pipeline to send an automatic email to any prospect coming to the website. Maybe the agent is handling customer support via an integration.

So if we revoke the access, there might be an impact on the business. So it's important to also understand what role that agent is playing in the business process.

So the speed of response is really dependent on whether you've built for it from the start. If you actually prepared yourself — many people won't have done that.

And that brings me to Proton Pass specifically, which obviously is the project which you lead on.

For someone who's heard all of this and actually wants to act upon this problem, how does Proton Pass help? What does it give you that just being more careful wouldn't give you?

So being more careful is something that everyone should do, but more often than not, people forget to be careful when under pressure, when there's FOMO involved, when they have to do things very quickly, or maybe they don't have the technical knowledge to do what careful means actually.

So that's what I mean by that — discipline doesn't really scale. So we need some structures to allow people to be careful, to be disciplined.

And LastPass or any password managers can be a good way to do that.

So we make sure that every credential is stored centrally so that admin can have an overview on what is stored in their company.

And then not use Slack or email to share username and password, because once it got out, it's very hard to know who has access to it.

And then anyone having access can use those credentials and we have no idea.

And if people are technical, then it's better for them to, if they want to use a secret, they can reference the secret from a password vault instead of copy and pasting them directly into the tool.

It's going to work better, and a lot of tools support that by integration with the password managers to get a secret instead of you having to copy and paste the password into the tool.

And recently in ProtonPass, we also created a feature called AI access token that allows a human to create an access token that they're going to give to the AI, which access the AI will have exactly in their vault.

And then whenever AI wants to access something, AI has to give a reason — why do I want that?

If AI tries to access, let's say, your storage account, AI should give a reason like, because I want to upload the latest invoice, for example, and later on, human can see the timeline of the AI access and see the reason why it's trying to access something.

And this way, human can be informed of what AI is actually doing and maybe intervene when something abnormal happens.

So it's like an audit log in a way, isn't it? Fantastic.

So it's not just about having good intentions as a business — it's also about having the infrastructure to back all of these up.

So what I always like to do when I chat to vendors is try and find some actionable advice for our listeners.

If someone's listened to all of this and they want to do one thing this week, what would you tell them?

So I think the first thing to do is to make the inventory, to list all the AI agents that you have enabled, and try to understand what they have access to and what the consequences would be if we remove them.

On top of that, it's better to tell everyone in the company to have some basic security practice, like never share passwords on Slack or email, have strong and unique passwords, enable two-factor authentication, etc.

I think with that, you can already improve a lot of your security posture.

Well, Son, this has been really interesting. Thank you so much for joining me on Smashing Security today.

And listeners, if you think that your firm needs a password manager built for business that doesn't compromise on security or slow your team down, then why not check out ProtonPass?

It's built on Swiss infrastructure, open-source architecture, and you can check out a free trial of ProtonPass for your business at proton.me/smashing. That's proton.me/smashing.

Thanks so much, Son, for joining us on this week's show. Well, that just about wraps up the show for this week. Thank you so much, Duck, for joining us.

I'm sure lots of our listeners would love to find out what you're up to and follow you online.

What's the best way to do that?

The best way is to go to my own website, that is paulducklin.com/about, and if you would like to read a lot of articles that I have been writing lately, you can go to one of my customers' websites where I do a lot of deep dive technical articles that you mentioned already, and that is solcyber.com/blog.

Terrific stuff.

And of course, Smashing Security is on social media as well. You can find it on Blue Sky and on Reddit and on Mastodon.

You can also find me, Graham Cluley, up there and on LinkedIn as well. And don't forget to ensure you never miss another episode.

Follow Smashing Security in your favourite podcast app, such as Apple Podcasts, Spotify, and Pocket Casts.

For episode show notes, sponsorship info, guest and the entire back catalog of 472 episodes, check out smashingsecurity.com. Until next time, cheerio.

You've been listening to Smashing Security with me, Graham Cluley.

I'm ever so grateful to Paul Ducklin for joining us this week and to this episode's sponsor, ProtonPass, Vanta, and CoreView.

And also, of course, tremendous thanks to our Patreon supporters.

This week we are pulling out of the hat for special mention the following patrons: Cory, Alex Tasker — I imagine they're very good at to-do lists — Bree Bustle, who is quite possibly the principal dancer at the Royal Ballet, Ted Wilkinson — sounds like the kind of reliable fellow you'd trust for a double glazing recommendation — Matt H, Dimitri, Alexander Hugues, back again, still sounding very grand, probably has a wonderfully long driveway.

Skadone, all lowercase, absolutely no time for capitals, far too busy. Butterfly, who's drifted in on gossamer wings, and SK, just the two initials, very mysterious.

Thank you all so much, you are wonderful.

Those are just a few members of Smashing Security Plus, our community, which gets their episodes ad-free and earlier than the general public.

And they can also have the privilege of having their names pulled out of a hat at random to be mocked at the end of the show.

If you'd fancy a little bit of that, all you have to do is join Smashing Security Plus.

Just head over to smashingsecurity.com/plus for all the details where you can become a patron of the show.

But you can also support the show in plenty of other ways that don't cost a penny. You can like, you can subscribe, you can leave a 5-star review, you can spread the word.

Go on, tell your friends about Smashing Security and your enemies. In fact, tell everybody, why not? Just go for it. Every little bit helps and I really, really appreciate it.

Well, thank you for listening this week and I hope you will tune in to our future episodes as well. Until then, cheerio, bye-bye.